On October 13, 2016, we were warned of the 115-year limit to our lifespans. It was a spoiler for those of us certain we could live forever and contestable news for researchers worldwide.
The research was published in Nature and hotly debated the moment it hit newsstands. The first to write an open review on Publons was James Vaupel, a demographist cited in the paper from Duke University. He was followed by Stanford University’s James Fries, also cited, before the authors – and a tide of expert reviewers – entered the discussion to weigh in.
The review was soon brimming with new information about the study’s much-debated human ‘outliers.’ Reviewers added corrections to the research, so-called improvements to the data, and alternative approaches to the hypothesis.
It quickly grew into a document that citation counts would take years to catch up on – and in doing so essentially sped up the research process.
The Pre-Publication Black Box
The lifespan study’s lengthy reviews helped bring to light what’s often missing in newly released research: a transparent evaluation process.
A manuscript’s ‘journey’ before publication is rarely made public. We often lose all the information involved in the decision-making process: we often don’t know who the reviewers are, if multiple revisions were made, why the decision to publish was made – and what information was used to make that call.
As Nikolaus Kriegeskorte puts it:
“The detailed evaluations are kept secret and contribute to the reception of a paper only after being reduced to a categorical quality stamp: the journal label. This constitutes a loss to the scientific community and to the general public of valuable judgements that are already being performed and paid for.”
Pre-publication review is the gold standard in protecting the quality and integrity of research, and key to helping stamp out fraudulent, careless and misleading results. But it’s secretive nature does come at a cost for readers in today’s publish or perish culture.
The unrelenting pressure to publish new research has not only overburdened researchers and reviewers – it is damaging our ability to decipher some of the most important studies today.
Post-publication peer review goes a long way to remedying that.
Whether signed or anonymous (and for context, 77% of the post-publication reviews on Publons are signed), these reviews are invaluable insights into the minds of experts. As readers we’re privy to the constructive criticism and collaboration amongst the experts, reviewers, and authors in the field. This not only guides our understanding of the study in a wider context, it can direct our attention to some of the most important research in any given discipline.
And while it’s true the post-publication peer review process is not flawless (John Tennant offers a nice writeup on its barriers), there’s no denying the practice enables more transparency, accountability, discussion, and better, faster research.
Gaetan Burgio’s failure to reproduce this CRISPR-cas9 study, and Giovanni Marco Dall’Olio’s follow-up research to a cancer cell study, are two reviews providing evidence of this.
Celebrating Transparency, Honouring Reviewers
We know the value of post-publication peer review here at Publons, and over the years have created tools and features to speed up and improve the process.
For example, reviewers can ‘score’ the quality and significance of a paper out of 10 to give readers instant insights into the research in question. They can also generate DOIs for reviews endorsed by the Publons community, giving it more weight as a research output.
We also wanted a way to formally acknowledge these contributions to research, so we named our top post-publication peer reviewers in our Publons Peer Review Awards.*
These awards celebrate all reviewers and editors as the sentinels of science and research, keeping watch over the quality and integrity of scholarly communication.
Thanks to their critical eye and devotion to sound research, we continue to expand the sphere of human knowledge. And thanks to post-publication peer reviewers especially, we can not only broaden our understanding of research, but see the mechanisms and discussions that improve it.
The work they do benefits everyone – and that deserves recognition.
5 steps to writing a winning post-publication peer review
“The best post-publication reviews should do five things,” says Oxford University’s James Buchanan.
With all of the above in mind, we asked James, a health economist who has written a great series of post-publication reviews on genome sequencing, for some top tips to writing them.
He told us that the key thing a post-publication peer reviewer should bear in mind when writing a review is to make sure that whoever reads it has a better understanding of the journal article at the end of the review. The most important thing to avoid doing, he said, is offering unconstructive and overly confrontational criticism.
Here’s five steps he suggests you take:
- Tell the reader why you have undertaken the review – what has motivated you to give up your evening or weekend to write about this paper?
- Critically and constructively appraise the journal article, highlighting key findings and limitations in a user-friendly manner.
- Place the research findings in context and help the reader to appreciate the relative importance of the article.
- A good post-publication review could (should?) add new information that helps readers to understand or make use of the research findings – that can be your fourth step.
- Finally, a post-publication review should be publicly available and, ideally, signed, as this often makes arguments more compelling to the reader.
Give it a go today. Write a post-publication peer review on Publons and get into the running to be named a Sentinel of Science and Research.
*Our Publons Peer Review Awards honor the highest achievers in peer review across all academic disciplines. We’ve added a number of new categories this year, including the top post-publication peer reviewers, and the Sentinel Award – for outstanding advocacy, innovation or contribution to scholarly peer review. Sponsors include Sage, Wiley, Wolters Kluwer, Catalyst Grant and Web of Science, and it is open to all reviewers on Publons.
