6 Common Flaws To Look Out For in Peer Review

It’s not always easy to spot flaws in research papers. Sometimes an error is glaringly obvious - like a vague abstract with no aim and little data - and other times it’s like finding a needle in a stack of pins. (Like, really, really sharp pins that leave you dreaming of haystacks). Luckily, the solution isn’t all that prickly. The trick is knowing what to look for during peer review, where to find it and, importantly, how severe the error is.

To help with this, we’ve pulled together a list of six common flaws you can watch out for as a reviewer.

This isn’t an extensive list; research errors differ for every field as do the types of studies conducted. It is a helpful starting point, however, that will enable you to guard against the more common mistakes made in a manuscript. Once you start accepting more invites to review and become more confident reading a manuscript critically, you can build on this list with more specialised examples.

See here for our 12 step guide on reviewing a manuscript critically.

The benefit of learning these common flaws is two-fold. As a peer reviewer, you’ll not only play a vital role in protecting the quality and integrity of scientific research, you’ll learn how to avoid the same errors in your own work, inevitably increasing your chances of getting published.

Your manuscripts will also improve because, over time, you’ll learn how to fine-tune your own manuscript. Peer reviewing will help you evaluate the importance and accuracy of your research question; the appropriateness of methodological and statistical approaches; and build up a set of best-practice tips to prepare and organise your research project. As an early career researcher, peer review will also help you forge those much-needed connections with editors and experts in your field of study. It’s win-win.

So, what should you watch out for to help protect research and become a great peer reviewer?

Oh, you know, start off with these six common flaws and soon you'll be all like...

6 Common Flaws in a Manuscript



1. Inappropriate study design for the study aims


A study’s design is crucial to obtaining valid and scientifically sound results. Familiarise yourself with those commonly used in your field of research. If you come across an uncommon study design, read the researchers’ use and justification of it carefully, and question how it might affect their data and analysis. Review the study design critically but also remember to be open-minded. Just because something is new and unfamiliar it does not automatically mean it is incorrect or flawed.


2. Deviating from standard/best practice and methodologies


Similar to the above. The methods section, for example, should explain the steps taken to produce the results. If these are not clear or you’re left questioning their validity, it’s important to make your concerns known. And if they are unusual then, as with the study design, examine the researchers’ justification carefully with the view to ask more questions if necessary. Nonacademic discourse is another deviation from best practice whereby opinionated and biased statements are used throughout the study.


3. Over-interpretation of results


Over-interpretation has no place in research. Ensure the conclusions drawn in the paper are based on the data presented and are not extrapolated beyond that (to a larger population or ecological setting, for example). You should also watch out for studies that focus on seemingly important differences where none exist.


4. Commenting beyond the scope of the article


“That’s beyond the scope of this paper” is a common phrase in academic writing. As a reviewer, watch out for papers that include comments or statements not pertaining to the research project and data at hand.


5. Lack of evidence to support conclusions


A research paper’s concluding statements must be justified and evidence-based. If you’re not convinced of the results, it could mean the researchers need to clarify aspects of their methodological procedure, add more references to support their claims, or include additional data or further analysis.


6. Too many words


A common pain point in manuscripts is that it’s too wordy. It’s important to keep check on this scenario and encourage clear, concise and effective text where possible. Too many words can be distracting for the reader, which at best could cause them to lose interest and at worst could lead to them misinterpreting the research.


If you’re interested in learning more about common flaws and how to address them in your peer review, sign up for our Publons Academy. This is a free on-demand course that teaches you how to master the core competencies of peer reviewing, and to connect with editors at elite journals.


Needles photo: Di_Chap; Gif: Red Bull; Magnifying glass: Tall Chris Publons allows you to record, verify, and showcase your peer review contributions in a format you can include in job and funding applications (without breaking reviewer anonymity).