Double‐blind peer review affects reviewer ratings and editor decisions at an ecology journal

Abstract

Abstract There is substantial evidence that systemic biases influence the scholarly peer review process. Many scholars have advocated for double‐blind peer review (also known as double‐anonymous review) to reduce these biases. However, the effectiveness of double‐blind peer review in eliminating biases is uncertain because few randomized trials have manipulated blinding of author identities for journal submissions and those that have are generally small or provide few insights on how it influences reviewer biases. In 2019, Functional Ecology began a large, randomized trial, using real manuscript submissions, to evaluate the various consequences of shifting to double‐blind peer review. Research papers submitted to the journal were randomly assigned to be reviewed with author identities blinded to reviewers (double‐blind review) or with authors identified to reviewers (single‐blind review). In this paper, we explore the effect of blinding on the outcomes of peer review, examining reviewer ratings and editorial decisions, and ask whether author gender and/or location mediate the effects of review type. Double‐blind review reduced the average success of manuscripts in peer review; papers reviewed with author identities blinded received on average lower ratings from reviewers and were less likely to be invited for revision or resubmission. However, the effect of review treatment varied with the author's location. Papers with first authors residing in countries with a higher human development index (HDI) and/or higher average English proficiency fared much better than those from countries with a lower HDI and lower English proficiency, but only when author identities were known to reviewers; outcomes were similar between demographic groups when author identities were not known to reviewers. Blinding author identities had no effect on gender differences in reviewer ratings or editor decisions. Our data provide strong evidence that authors from higher income and/or English‐speaking countries receive significant benefits (a large positive bias) to being identified to reviewers during the peer review process and that anonymizing author‐identities (e.g. double‐blind review) reduces this bias, making the peer review process more equitable. We suggest that offering optional blinding of author identities, as some journals allow, is unlikely to substantially reduce the biases that exist because authors from higher‐income and English‐speaking countries are the least likely to choose to be reviewed with their identity anonymized. Read the free Plain Language Summary for this article on the Journal blog.