Fragility of Statistically Significant Outcomes in Colonic Diverticular Disease Randomized Trials: A Systematic Review

Abstract

BACKGROUND: The p value has been criticized for an oversimplified determination of whether a treatment effect exists. One alternative is the fragility index. It is a representation of the minimum number of non-events that would need to be converted to events to increase the p value above 0.05. OBJECTIVE: To determine the fragility index of randomized controlled trials assessing the efficacy of interventions for patients with diverticular disease since 2010 to assess the robustness of current evidence. DATA SOURCES: MEDLINE, Embase, and CENTRAL were searched from inception to August 2022. STUDY SELECTION: Articles were eligible for inclusion if they were randomized trials conducted between 2010 and 2022 with parallel, superiority designs evaluating interventions in patients with diverticular disease. Only randomized trials with dichotomous primary outcomes with an associated p-value of less than 0.05 were considered for inclusion. INTERVENTION(S): Any surgical or medical intervention for patients with diverticular disease. MAIN OUTCOME MEASURES: The fragility index was determined by adding events and subtracting non-events from the groups with the smaller number of events. Events were added until the p-value exceeded 0.05. The smallest number of events required was considered the fragility index. RESULTS: After screening 1,271 citations, 15 randomized trials met inclusion criteria. Nine of the studies evaluated surgical interventions and six evaluated medical interventions. The mean number of patients randomized and lost to follow-up per RCT was 92 (SD 35.3) and 9 (SD 11.4), respectively. The median fragility index was 1 (range: 0-5). The fragility indices for the included studies did not correlate significantly with any study characteristics. LIMITATIONS: Small sample, heterogeneity, and lack of inclusion of studies with continuous outcomes. CONCLUSIONS: The randomized trials evaluating surgical and medical interventions for diverticular disease are not robust. Changing a single outcome event in most studies was sufficient to make a statistically significant study finding non-significant.