Effect of librarian collaboration on otolaryngology systematic review and meta-analysis quality

Abstract

Objective: To determine if librarian collaboration was associated with improved database search quality, search reproducibility, and systematic review reporting in otolaryngology systematic reviews and meta-analyses. Methods: In this retrospective cross-sectional study, PubMed was queried for systematic reviews and meta-analyses published in otolaryngology journals in 2010, 2015, and 2021. Two researchers independently extracted data. Two librarians independently rated search strategy reproducibility and quality for each article. The main outcomes include association of librarian involvement with study reporting quality, search quality, and publication metrics in otolaryngology systematic reviews and meta-analyses. Categorical data were compared with Chi-Squared tests or Fisher’s Exact tests. Continuous variables were compared via Mann Whitney U Tests for two groups, and Kruskal-Wallis Tests for three or more groups. Results: Of 559 articles retrieved, 505 were analyzed. More studies indicated librarian involvement in 2021 (n=72, 20.7%) compared to 2015 (n=14, 10.4%) and 2010 (n=2, 9.0%) (p=0.04). 2021 studies showed improvements in properly using a reporting tool (p<0.001), number of databases queried (p<0.001), describing date of database searches (p<0.001), and including a flow diagram (p<0.001). Librarian involvement was associated with using reporting tools (p<0.001), increased number of databases queried (p<0.001), describing date of database search (p=0.002), mentioning search peer reviewer (p=0.02), and reproducibility of search strategies (p<0.001). For search strategy quality, librarian involvement was associated with greater use of “Boolean & proximity operators” (p=0.004), “subject headings” (p<0.001), “text word searching” (p<0.001), and “spelling/syntax/line numbers” (p<0.001). Studies with librarian involvement were associated with publication in journals with higher impact factors for 2015 (p=0.003) and 2021 (p<0.001). Conclusion: Librarian involvement was associated with improved reporting quality and search strategy quality. Our study supports the inclusion of librarians in review teams, and journal editing and peer reviewing teams.