You, Too, Can Peer Review

Publishing research is crucial to a successful career in science. Knowing this, at some point, most scientists will embark on the process of submitting a manuscript for review at a journal. Equally as important, however, is the often overlooked opportunity to take a seat on the other side of the table. Learning how to serve as a manuscript reviewer can pay dividends in unexpected ways, not only by enhancing an ability to navigate the overall publishing process, but also by honing an ability to draft clearer and more concise articles communicating one’s own research. But to get there, potential peer reviewers must first gain a solid understanding of the process.

The purpose of peer review, explained Angela Welch, senior publisher of Elsevier Chemical and Biomedical Engineering, is to validate the research presented and to assess the originality of the work within the context of that field. “A main critique is that an author doesn’t demonstrate how [their finding] is better than the previous one,” she said to the evening’s audience, comprised largely of graduate and postdoctoral researchers. Along the way, a reviewer often provides valuable suggestions by recommending statistical methods, figure design techniques or relevant citations that clarify an author’s findings and enrich a discussion of the work.

First, however, that article must land in a reviewer’s hands—many don’t make it even that far. When editors receive any new manuscript submission, the first step is to screen it to determine whether it fits the interests of the journal. “More times than not, an initial rejection is because [the article] is out of scope for the journal,” Welch said. In this case, the editors may return the submission, possibly suggesting the authors transfer the manuscript to another, more appropriate journal. But if an article jumps through this initial hoop, the review process begins when at least two peers are selected to assess it. Typically, it takes these peer reviewers about four to six weeks to write their responses, which can include recommendations to the editors to accept the manuscript—either outright or pending revisions—or reject it. After that decision is made, the article, along with the reviewers’ comments, gets sent along to the authors, who can either accept or appeal the decision. This process continues for one to three rounds, until the manuscript is revised to the reviewers’ agreement, or the editors decide that the concerns have been addressed.

Welch stressed that fairness and impartiality on the part of the reviewers is key to the effectiveness of the peer review process. To support this, many journals prevent reviewers from communicating with each other or directly with authors. To maintain anonymity, three main blinding methods are employed: single blind (the most typical), in which the reviewers’ identities are unknown to the authors; double blind, in which reviewers also do not know the identities of the authors; and triple blind, in which even the editors are not revealed. However, the argument has been advanced that lifting this veil could actually increase the pressure on these individuals to be accountable, resulting in more unbiased reviews. In fact, some journals are now experimenting with “cross-reviewing,” in which reviewers can see what their counterparts have written about a manuscript, and “transparent reviewing,” in which reviews are even open to the public.

Pausing for a lively Q&A, Welch brought up Kam Leong, editor-in-chief of Biomaterials and a biomedical engineering professor at Columbia University, to help field questions.

“How do the editors of a journal select reviewers?” asked one audience member. Editors keep a list of researchers around the world, Leong replied, so the reviewers chosen are likely ones that have already published on the manuscript’s topic. In addition, reviewers are selected based on key words an author includes in the manuscript. “So that’s a lesson [for authors]: pick your keywords well,” he said. Welch added that journals create taxonomies of these terms to build profiles of researchers once they publish in a journal, and that these can be used to match a manuscript with the most appropriate reviewers.

“Peer reviewing is voluntary, so why do people do it?” another audience member asked. In most cases, people do it to stay up to date in their field, Welch replied. It also allows them to network for career development, since the experience affords them opportunities to develop a professional relationship with journal editors.

“How does someone interested in peer reviewing get involved?” wondered another attendee. The best way to get involved, Welch replied, is to go to conferences and meet journal editors. In addition, it helps to publish in the journal for which you’d like to edit, since that ensures you’ve already passed their standards test, and therefore have credibility to review the work of others. But researchers can also simply write to a journal to ask whether they’re currently seeking reviewers for any manuscripts in your field, she added. Finally, some journals are experimenting with new methods to recruit reviewers. For example, Biomaterials journal is conducting a pilot study inviting volunteers to go online and sign up to peer review.

In response to a query on how to determine which journal might be a good fit, Welch suggested that interested reviewers look at his or her own reference list. Knowing what journals you read would help to narrow down journals to ones you might want to review. Then, you might join a journal club on your campus that will help you to read more critically. “I find that a good reviewer is a good scientist,” Leong added. “So learning to be a good reviewer is worth it."

Next, Welch walked through the various sections of a manuscript, covering specific questions to keep in mind in seven main areas:

Title: What do you expect to learn from the article given this title? A reviewer should assess whether he/she was given the right context in the introduction and most importantly, that the significance of the contribution is made clear.

Abstract: Are you excited by the topic and does the text accurately highlight the main methods and results?

Results: Does the author lay out an analytical description of the data with minimal interpretation?

Figures and tables: Do these stand on their own so that a reader could understand the paper without reading the results?

Methods: Is the sample size large enough? Should any supplementary figures be included? Is the methodology description comprehensive?

Discussion and Conclusions: Does it connect out to the bigger picture without introducing unsupported statements? Often, this is the hardest part for authors to write, Welch said. A reviewer should advise on how authors can highlight the importance of the work and the impact it will have on the broader field of research without drawing conclusions that are too far-reaching or unjustified.

References: Are the citations recent? Are authors only citing their own work or are they bringing in the work of other researchers as well?

In addition to these questions, reviewers can visit the journal’s website where they can find guidelines for authors that are specific to that journal.

During the second half of the evening, a sample manuscript made the rounds for an interactive, mock peer review session. Audience members broke out into small groups, where Welch encouraged them to focus on formulating concrete statements citing examples, over simple, qualitative assessments. If audience members were actually preparing written comments, “we prefer reviewers to [distill their ideas down to] a bullet pointed list instead of narrative,” Leong said. Such attention to detail counts—authors aren’t the only ones being judged in the peer review process. Editors will assign reviewers a score 1 to 100, with the best ones hitting around 85 to 90, based on how clearly they articulate their comments.

Overall, responses should be broken down into a quick article synopsis, followed by a first of major and minor comments, and finally, a recommendation on whether to accept the manuscript—with or without revision—or reject it. And doing so, Welch advised, it’s good to keep in mind that “[t]he best reviewers are teachers, not critics.” For further references on peer reviewing, research writing, and publishing in general, she directed the audience to visit the online Elsevier Publishing Campus.

In fact, the most important aspect of a peer reviewer’s job is trying to help the authors. “Even if you suggest reject[ing] the paper,” she said, “Try to still provide in-depth constructive comments. These will be helpful to the authors to improve their work for a possible second attempt.”