An Assessment of Transparency and Reproducibility-related Research Practices in Otolaryngology

Abstract

ABSTRACTIntroductionClinical research serves as the foundation for evidence-based patient care, and reproducibility of results is consequently critical. We sought to assess the transparency and reproducibility of research studies in otolaryngology by evaluating a random sample of publications in otolaryngology journals between 2014 and 2018.MethodsWe used the National Library of Medicine catalog to identify otolaryngology journals that met the inclusion criteria (available in the English language and indexed in MEDLINE). From these journals, we extracted a random sample of 300 publications using a PubMed search for records published between January 1, 2014, and December 31, 2018. Specific indicators of reproducible and transparent research practices were evaluated in a blinded, independent, and duplicate manner using a pilot-tested Google form.ResultsOur initial search returned 26,498 records, from which 300 were randomly selected for analysis. Of these 300 records, 286 met inclusion criteria and 14 did not. Among the empirical studies, 2% (95% CI, 0.4%-3.5%) of publications indicated that raw data were available, 0.6% (95% CI, 0.3%-1.6%) reported an analysis script, 5.3% (95% CI, 2.7%-7.8%) were linked to an accessible research protocol, and 3.9% (95% CI, 1.7%-6.1%) were preregistered. None of the publications had a clear statement claiming to replicate, or to be a replication of, another study.ConclusionsInadequate reproducibility practices exist in otolaryngology. Nearly all studies in our analysis lacked a data or material availability statement, did not link to an accessible protocol, and were not preregistered. Most studies were not available as open access. Taking steps to improve reproducibility would likely also improve patient care.