Challenging the Concept of Statistical Fragility: Is There Any Value Added?

Abstract

Abstract: Today, well-designed randomized clinical trials (RCTs) are considered the pinnacle of clinical research, and they inform many practices in orthopaedics. When designing these studies, researchers conduct a power analysis, which allows researchers to strike a balance between (1) enrolling enough patients to detect a clinically important treatment effect (i.e., researchers can be confident that the effect is unlikely due to chance) and (2) cost, time, and risk to patients, which come with enrolling an excessive number of patients. Because researchers will have a desire to conduct resource-efficient RCTs and protect patients from harm, many studies report a p value that is close to the threshold for significance. The concept of the fragility index (FI) was introduced as a simple way to interpret RCT findings, but it does not account for RCT design. The adoption of the FI conflicts with researchers’ goals of designing efficient RCTs that conserve resources and limit ineffective or harmful treatments to patients. The use of the FI may reflect many clinicians’ lack of familiarity with interpreting p values beyond “significant” or “nonsignificant.” Instead of inventing new metrics to convey the same information provided by the p value, greater emphasis should be placed on educating clinicians on how to interpret p values and, more broadly, statistics, when reading scientific studies.