How to critically evaluate a manuscript: 12 questions you should always ask yourself

It's finally happened: you have received your first invitation to peer review. You accept, pick up your red pen, and shuffle gleefully in your chair. This is your much anticipated contribution to the scientific community. But then the panic sets in: what does peer review really mean, and what should you look out for while reading the manuscript?

Peer review can be challenging for new academics. Your role is to help maintain the quality and integrity of published research and, in turn, protect the public from flawed and misleading findings. This may feel like a daunting task given the admissions of fraudulent research practices, surge in retractions and the reproducibility crisis facing science today - but fighting against these problems is not only vital for scholarly communication, it will also improve your own skills as a researcher.

Your peer review contributions will help you understand what editors are looking for, and you’ll become a better writer and a more successful published author in the process. You’ll keep abreast of research in your field, learn new and best-practice methods, and start examining your own research from that critical vantage point.

However, to start contributing to the scientific community and reaping the benefits of peer review, you’ll first need to learn what makes a great peer reviewer.

Key to this is knowing when to accept a review invitation (see here for our tips) and how to critique a research paper.

We asked Publons Academy Advisor Elisabeth Bik for her advice on how to read a manuscript critically. Bik is a Research Associate at the Department of Microbiology and Immunology at Stanford University's School of Medicine. Alongside putting together a helpful list of general questions to ask yourself while reviewing, Bik offered her valuable expertise during the creation of our Publons Academy. This is a free on-demand course that teaches you how to become a better peer reviewer. You can sign up for that here to master the core competencies of peer reviewing and to connect with editors at elite journals.

Top Tips to Critically Review a Research Paper

Elisabeth Bik: Asking these questions should help you form an opinion about the paper, even if you have no idea where to start. It's the list that I wished I had access to when I started my first peer review. Here we go:

1. Do you have a conflict of interest when reviewing this paper? Do you collaborate with these authors, are they your personal friends, or are they direct competitors? Have you reviewed (and rejected) this paper before? If so, you need to let the editors know.

2. Do the title and abstract cover the main aspects of the work, would it spark interest to the right audience?

3. Is the Introduction easy to follow for most readers of this particular journal? Does it cite the appropriate papers? Does it provide a hypothesis or aim of the study?

4. Does the Methods section provide enough details for the general reader to repeat the experiments?

5. If you skip the Methods, does the Results section give the right amount of detail to understand the basic details of the experiments?

6. Do the Results refer to the figures in a logical order? Do the numbers in the tables add up correctly? Are any figures/tables mislabeled or unclear?

7. Given the data that was obtained in this study, did the authors perform all the logical analyses? Did they include the proper controls?

8. Does the Discussion address the main findings, and does it give proper recognition to similar work in this field?

9. In general, is the paper easy to follow and does it have a logical flow? Are there any language issues?

10. Did the authors make all their data (e.g. sequence reads, code, questionnaires used) available for the readers?

11. Is this paper novel and an advancement of the field, or have other people done very similar work?

12. Finally (and hopefully you will never have to answer yes to any of these questions): Does the paper raise any ethical concerns? Any suspicion of plagiarism (text or experiments), duplicated or tampered images, lack of IRB approval, unethical animal experiments, or "dual use of research concern"?

Follow these tips and you’ll be well on your way to becoming a skilled and sought-after peer reviewer. And remember - if you want to master peer review and connect with top editors then head over to our Publons Academy here.

The Publons Academy is a free online course designed by expert reviewers, editors and Nobel Prize winners to teach you how to become a master of peer review and connects you with real life editors.