Peer Review in a General Medical Research Journal Before and During the COVID-19 Pandemic

Abstract

ImportanceAlthough peer review is an important component of publication for new research, the viability of this process has been questioned, particularly with the added stressors of the COVID-19 pandemic.ObjectiveTo characterize rates of peer reviewer acceptance of invitations to review manuscripts, reviewer turnaround times, and editor-assessed quality of reviews before and after the start of the COVID-19 pandemic at a large, open-access general medical journal.Design, Setting, and ParticipantsThis retrospective, pre-post cohort study examined all research manuscripts submitted to JAMA Network Open between January 1, 2019, and June 29, 2021, either directly or via transfer from other JAMA Network journals, for which at least 1 peer review of manuscript content was solicited. Measures were compared between the period before the World Health Organization declaration of a COVID-19 pandemic on March 11, 2020 (14.3 months), and the period during the pandemic (15.6 months) among all reviewed manuscripts and between pandemic-period manuscripts that did or did not address COVID-19.Main Outcomes and MeasuresFor each reviewed manuscript, the number of invitations sent to reviewers, proportions of reviewers accepting invitations, time in days to return reviews, and editor-assessed quality ratings of reviews were determined.ResultsIn total, the journal sought review for 5013 manuscripts, including 4295 Original Investigations (85.7%) and 718 Research Letters (14.3%); 1860 manuscripts were submitted during the prepandemic period and 3153 during the pandemic period. Comparing the prepandemic with the pandemic period, the mean (SD) number of reviews rated as high quality (very good or excellent) per manuscript increased slightly from 1.3 (0.7) to 1.5 (0.7) (P < .001), and the mean (SD) time for reviewers to return reviews was modestly shorter (from 15.8 [7.6] days to 14.4 [7.0] days; P < .001), a difference that persisted in linear regression models accounting for manuscript type, study design, and whether the manuscript addressed COVID-19.Conclusions and RelevanceIn this cohort study, the speed and editor-reported quality of peer reviews in an open-access general medical journal improved modestly during the initial year of the pandemic. Additional study will be necessary to understand how the pandemic has affected reviewer burden and fatigue.