Promoting Reproducibility and Integrity in Observational Research: One Approach of an Epidemiology Research Community

Abstract

To increase research reproducibility, sharing of study data, analysis code, and use of standardized reporting are increasingly advocated. However, beyond reproducibility, few initiatives have addressed the integrity of how research is conducted before manuscripts are submitted. We describe a decades-long experience with a comprehensive approach based in an academic research community around prospective cohort studies that is aimed at promoting a culture of integrity in observational research. The approach includes prespecifying hypotheses and analysis plans, which are discussed in the research community and posted; presentation and discussion of analysis results; mandatory analysis code review by a programmer; review of concordance between analysis output and manuscripts by a technical reviewer; and checks of adherence to the process, including compliance with institutional review board requirements and reporting stipulations by the National Institutes of Health. The technical core is based in shared computing and analytic environments with long-term archiving. More than simply a list of rules, our approach promotes research integrity through integrated educational elements, making it part of the “hidden curriculum,” by fostering a sense of belonging, and by providing efficiency gains to the research community. Unlike reproducibility checklists, such long-term investments into research integrity require substantial and sustained funding for research personnel and computing infrastructure. Our experiences suggest avenues for how institutions, research communities, and funders involved in observational research can strengthen integrity within the research process.