How to Write an Effective Referee Report and Improve the Scientific Review Process

Abstract

The review process for academic journals in economics has grown vastly more extensive over time. Journals demand more revisions, and papers have become bloated with numerous robustness checks and extensions. Even if the extra resulting revisions do on average lead to improved papers—a claim that is debatable—the cost is enormous. We argue that much of the time involved in these revisions is a waste of research effort. Another cause for concern is the level of disagreement amongst referees, a pattern that suggests a high level of arbitrariness in the review process. To identify and highlight what is going right and what is going wrong in the reviewing process, we wrote to a sample of former editors of the American Economic Review, the Journal of Political Economy, the Quarterly Journal of Economics, Econometrica, the Review of Economic Studies, and the Journal of Financial Economics, and asked them for their thoughts about what might improve the process. We found a rough consensus that referees for top journals in economics tend to make similar, correctable mistakes. The italicized quotations throughout this paper are drawn from our correspondence with these editors and our own experience. Their insights are consistent with our own experiences as editors at the Journal of Finance and the Review of Financial Studies. Our objective is to highlight these mistakes and provide a roadmap for how to avoid them.