Vygintas Aliukonis

30 October 2020

No Comments

Home Op-ed

Peer-review crisis: reviewer fatigue.

Peer-review crisis: reviewer fatigue.

Let us start from a provocative comparison made by SirPeter Lachmann who claimed that “Peer review is to science what democracy is to politics. It’s not the most efficient mechanism, but it’s the least corruptible[i]. So, why nowadays peer-review is receiving progressively more criticism, and studies are questioning the effectiveness and viability of the approach[ii]? Furthermore, why in a time of globalization, when competition between researchers grew from national to international, researchers are refusing to become peer reviewers?

 

One of the reasons researchers decline to review the papers submitted to them is that the article’s ideas do not relate to the course of their reaserch field, which does not allow for a qualitative assessment of other researchers’ achievements[iii][iv].

Peer review is to science what democracy is to politics. It’s not the most efficient mechanism, but it’s the least corruptible.

However, this is not the only reason for refusal:

In a 2015 survey of American scientists looking for reasons academics refuse to be reviewers, as many as 38% of those surveyed said they were just “too busy”.[v]Authors of the survey suggest that an answer is likely to be a direct consequence of a “reviewer fatigue”.

 

‘Reviewer fatigue’ is defined as the difficulty that an editor faces in recruiting reviewers, who may feel overwhelmed by receiving excessive invitations to evaluate manuscripts [vi]

 

Even though peer-review is one of the academics’ works, there is growing dissatisfaction with the fact that commercial publishers receive financial benefits from articles. In contrast, researchers who rate them only receive “honor points”.[vii]

Reviewer fatigue’ is defined as the difficulty that an editor faces in recruiting reviewers, who may feel overwhelmed by receiving excessive invitations to evaluate manuscripts.

This situation leads to more significant problems. Publishers, unable to find willing reviewers for  submitted paper, generally refuse to publish it not because of flaws in the article, but precisely because there was no way to ensure the peer review process[viii]. Unfortunately, the lack of reviewers prepared to evaluate submissions is not a new case. Some studies show this to be one of the reasons for the final decision to accept or reject an article for publication.[ix]

 

Assessing the situation makes it clear that it is imperative to encourage reviewers to maintain ever-increasing demand.

 

Some editors suggest that simply paying the reviewers would do the trick.[x][xi]And although few, there are already journals that apply such a model, such as the American Economic Association journals.

Unfortunately, the lack of reviewers prepared to evaluate submissions is not a new case.

Critics says that paying reviewers would on the one hand, raise the number of people able to rate submissions, but on the other hand, it raises doubts whether the reviewerswill not attempt to assess all requests received, simply to increase their income. The reviewer may not have sufficient time to perform the work assigned to him competently[xii]. From an ethical point of view, suggestions like these call into question the reviewer’s intrinsic motivation[xiii]and volunteering principles[xiv].

 

In addition to direct financial incentives, some authors and publishers offer other instruments. These techniques should not have drawbacks inherent from direct financial incentives[xv]. Examples of non-monetary rewards are free access to publishers’ journals, discounts on their products, or even reviewers’ certificates[xvi].

 

Some journals, such as the Journal of Clinical Epidemiology, collect ‘reviewers of the year”[xvii], thus encouraging their experts and helping scientists to make their name recognized.

The reviewer may not have sufficient time to perform the work assigned to him competently.

In general, the public recognition of the reviewer and the display of the name are well received by both – the reviewers and the authors. It is then debated whether adding the titles of reviewers to the article will not compromise reviewers’ freedom to be strict and encourage them to be too lenient[xviii].

 

Despite offering non-monetary rewards appear to be an exciting strategy for engaging reviewers.

 

However, this method of attracting reviewers is not extensively studied. A Romanian journal recently published a survey and analysis explicitly dedicated to the non-monetary practice showed that the approach is not only ineffective but, in some cases, even discourages the most voluntary motivated reviewers.[xix]

It is then debated whether adding the titles of reviewers to the article will not compromise reviewers’ freedom to be strict and encourage them to be too lenient.

Finally, for decades, researchers have relied on data from peer-reviewed journals, but peer review is a slow, difficult-to-change process and is now facing a human resources crisis.

 

None of the reviewer-promoting methods now known is sufficient enough to solve problems by applying it alone.

 

Therefore, it is necessary to encourage editors to use a combination of all these methods, thus ensuring an appropriate number of willing and qualified experts for reviewing scholarly journal submissions efficiently and qualitatively.

 

Overall, if academics are getting less inclined to accept the “burden of peer-reviewing,” who will review our manuscripts?

[i]Lachmann P. The research integrity initiative: progress report. Cope Report 2002:11.

[ii]Mulligan A. Is peer review in crisis. Oral Oncol. 2005 Feb;41(2):135-41.

[iii]Publons. Clarivate Analytics. In: Global State of Peer Review. Philadelphia, PA: Clarivate Analytics; 2018

[iv]Vesper I. Peer reviewers unmasked: largest global survey reveals trends. Nature 2019. Available at nature.com.

[v]Breuning M, Backstrom J, Brannon J, Gross BI, Widmeier M. Reviewer fatigue? Why scholars decline to review their peers’ work. PS Political Sci Politics 2015;48:595e600

[vi]Publons. Clarivate Analytics. In: Global State of Peer Review. Philadelphia, PA: Clarivate Analytics; 2018

[vii]Fox CW, Albert AYK, Vines TH. Recruitment of reviewers is becoming harder at some journals: a test of the influence of reviewer fatigue at six journals in ecology and evolution. Res Integr Peer Rev 2017;2:3

[viii]Ellwanger JH, Chies JAB. We need to talk about peer-review-Experienced reviewers are not endangered species, but they need motivation. J Clin Epidemiol. 2020 Sep;125:201-5

[ix]Fox CW. Difficulty of recruiting reviewers predicts review scores and editorial decisions at six journals of ecology and evolution. Scientometrics 2017;113:465e77

[x]Chang, J., & Lai, C. (2001). Is It Worthwhile to Pay Referees? Southern Economic Journal, 68(2), 457-463. doi:10.2307/1061605

[xi]Thompson, Gary D., Aradhyula, Satheesh V., Frisvold, George, Tronstad, Russell (2010), Does Paying Referees Expedite Reviews?: Results of a Natural Experiment. Southern Economic Journal: January 2010, Vol. 76, No. 3, 678– 692.

[xii]Tite L, Schroter S. Why do peer reviewers decline to review? A Survey. J Epidemiol Community Health 2007;61:9e12.

[xiii]Gagné, M. and Deci, E.L. (2005), Self‐determination theory and work motivation. J. Organiz. Behav., 26: 331-362. doi:10.1002/job.322

[xiv]Gneezy, Uri, Stephan Meier, and Pedro Rey-Biel. 2011.     “When and Why Incentives (Don’t) Work to Modify Behavior.” Journal of Economic Perspectives, 25 (4):  191-210. DOI: 10.1257/jep.25.4.191

[xv]Gallus, J. and Frey, B.S. (2016), Awards: A strategic management perspective. Strat. Mgmt. J., 37: 1699-1714. doi:10.1002/smj.2415

[xvi]Gasparyan AY, Gerasimov AN, Voronov AA, Kitas GD. Rewarding peer reviewers: maintaining the integrity of science communication. J Korean Med Sci 2015;30:360e4

[xvii]Journal of Clinical Epidemiology. Reviewers of the year – reviewer Award. J Clin Epidemiol 2018;104:146

[xviii]Nature Editorial. Three-year trial shows support for recognizing peer reviewers. Nature 2019;568:275

[xix]Zaharie MA, Seeber M. Are non-monetary rewards effective in attracting peer reviewers? A natural experiment. Scientometrics 2018;117:1587e609

Leave a Reply

Your email address will not be published. Required fields are marked *