Making research evaluation more transparent: Aligning research philosophy, institutional values, and reporting

Abstract

There is a growing interest in changing the culture of psychology to improve the quality of our science. At the root of this interest is concern over the reproducibility of key findings. A variety of large-scale replication attempts have revealed that a number of previously published effects cannot be reproduced, while other analyses indicate that the published literature is rife with underpowered studies and publication bias. These revelations suggest that it is time to change how psychological science is carried out and increase transparency of reporting. We argue change will be slow until institutions adopt new procedures for evaluating scholarly activity. We consider three actions that individuals and departments can take to facilitate change throughout psychological science. These three actions are the development of individualized research philosophy statements, the creation of an annotated curriculum vitae to improve the transparency of scholarly reporting, and the use of a formal evaluative system that explicitly captures behaviors that support reproducibility. Our recommendations build on proposals for open science by enabling researchers to have a voice in articulating (and contextualizing) how they would like their work to be evaluated and by providing a mechanism for more detailed and transparent reporting of scholarly activities.