Systematic Evaluation of the Robustness of the Evidence Supporting Current Guidelines on Myocardial Revascularization Using the Fragility Index

Abstract

Background: RCTs (randomized controlled trials) are the preferred source of evidence to support professional societies’ guidelines. The fragility index (FI), defined as the minimum number of patients whose status would need to switch from nonevent to event to render a statistically significant result nonsignificant, quantitatively estimates the robustness of RCT results. We evaluate RCTs supporting current guidelines on myocardial revascularization using the FI and FI minus number of patients lost to follow-up. Methods and Results: The FI and FI minus number of patients lost to follow-up of RCTs supporting the 2012 American College of Cardiology/American Heart Association Guideline for the Diagnosis and Management of Patients with Stable Ischemic Heart Disease, the 2014 Focused Update of the American College of Cardiology/American Heart Association Guideline for the Diagnosis and Management of Patients with Stable Ischemic Heart Disease, and the 2018 European Society of Cardiology/European Association for Cardio-Thoracic Surgery Guidelines for Myocardial Revascularization were calculated. Of 414 RCTs identified, 160 were eligible for FI calculation. The median FI was 8.0 (95% CI, 5.0–9.0) and the median FI minus number of patients lost to follow-up was 1.0 (95% CI, 0.0–3.0). FI was ≤3, indicating very limited robustness, in 44 (27.5%) RCTs, and was lower than the number LTF, indicating limited robustness, in 68 (42.5%) RCTs. FI was significantly (all P <0.05) correlated with the sample size, number of events, statistical power, journal impact factor, use of intention-to-treat analysis, and of composite end points and negatively correlated with the use of percutaneous interventions in the treatment arm and the P -value level. Conclusions: More than a quarter of RCTs that support current guidelines on myocardial revascularization have a FI of 3 or lower, and over 40% of trials reveal a FI that is lower than the number of patients lost to follow-up. These findings suggest that the robustness of the findings that support current myocardial revascularization guidelines is tenuous and vulnerable to change as new evidence from RCTs appears.