“The Grievance Studies Affair” Project: Reconstructing and Assessing the Experimental Design

Abstract

Recently, high media visibility was reached by an experiment that involved “hoaxlike deception” of journals within humanities and social sciences. Its aim was to provide evidence of “inadequate” quality standards especially within gender studies. The article discusses the project in the context of both previous systematic studies of peer reviewing and scientific hoaxes and analyzes its possible empirical outcomes. Despite claims to the contrary, the highly political, both ethically and methodologically flawed “experiment” failed to provide the evidence it sought. The experiences can be summed up as follows: (1) journals with higher impact factors were more likely to reject papers submitted as part of the project; (2) the chances were better, if the manuscript was allegedly based on empirical data; (3) peer reviews can be an important asset in the process of revising a manuscript; and (4) when the project authors, with academic education from neighboring disciplines, closely followed the reviewers’ advice, they were able to learn relatively quickly what is needed for writing an acceptable article. The boundary between a seriously written paper and a “hoax” gradually became blurred. Finally (5), the way the project ended showed that in the long run, the scientific community will uncover fraudulent practices.