An analysis of research quality underlying IDSA clinical practice guidelines: a cross-sectional study

Abstract

Abstract Context As a result of new developments in medicine, the need for evidence-based clinical practice guidelines (CPG) is of utmost importance. However, studies have shown that many medical societies are using low quality research to develop CPGs. Objectives To evaluate the quality of research underlying the CPGs issued by the Infectious Diseases Society of America (IDSA). Methods We examined 29 CPGs issued between January 1, 2012 and December 31, 2019 and classified each by research quality according to levels reported by the CPG authors and previously specified by the IDSA: Levels I through III, corresponding to high, moderate, and low quality of evidence, respectively. Each ranking was cross-checked with a second researcher to improve inter-rater reliability. To analyze evolution of research quality over time, three updated CPGs were randomly selected and compared to their original versions. Chi-square analysis was then performed to determine statistical significance. Results We evaluated the quality of research for 2,920 recommendations within the 29 CPGs that met our criteria and found that 418 (14%) were developed using high-quality (Level I) research from randomized, controlled trials. Of the remaining recommendations, 928 (32%) were based on moderate quality research (observational studies) and 1574 (54%) on low quality research (expert opinion). A Pearson chi-squared analysis indicated no-statistically significant difference between original guidelines or their subsequent updates for Clostridium difficile (χ2=0.323; n=85; degrees of freedom [df]=2; p=0.851), candidiasis (χ2=4.133; n=195; df=2; p=0.127), or coccidiomycosis (χ2=0.531; n=95; df=1; p=0.466). Conclusions The proportion of high-quality research underlying guideline recommendations is remarkably low, indicating that moderate and low quality evidence is still influencing infectious disease guidelines despite IDSA standards. Moreover, the quality of research has not significantly changed over time. IDSA CPGs are a formidable source of information for clinicians, but an increased number of quality studies should be utilized to further guide CPG development.