The ACM SIGSOFT Paper and Peer Review Quality Initiative

Abstract

Scholarly peer review is crucial to science: it not only determines what is published where, but also, indirectly, who is hired, funded and promoted. Yet, virtually every academic has peer review horror stories. Empirical evidence suggests that "peer review is prejudiced, capricious, inefficient, ineffective, and generally unscientific" [1]. An experiment at a major machine learning conference found that peer review was unreliable highlighted that the outcome of peer review can be very noisy [2, 3]. In May 2019, ACM SIGSOFT launched an initiative to improve the quality of research papers and peer reviews at software engineering venues. It has two main components: empirical standards and recommendations for improving review processes.