The Robustness of Trials That Guide Evidence-Based Orthopaedic Surgery

The fragility index (FI) may prove to be a powerful metric of trial robustness. The FI is the minimum number of patient events that would need to become nonevents in order to nullify a significant result. The fragility quotient (FQ) is the FI divided by the total sample size. This study evaluates the robustness of the 20% of orthopaedic clinical trials that were cited as having strong evidence in the American Academy of Orthopaedic Surgeons (AAOS) Clinical Practice Guidelines and that could be analyzed with these indices.

From the AAOS recommendations with strong evidence, we extracted the randomized controlled trials that were cited as having supporting evidence that could be analyzed with the FI. Each trial's FI was calculated using the fragility calculator (http://www.fragilityindex.com). With use of the Cochrane Risk of Bias Tool 2.0, we evaluated the likelihood of bias. We also performed a post hoc power analysis of eligible studies.

The median FI for the 72 trials was 2 events, and the median FQ was 0.022. Of the 72 trials, only 3 (4.2%) were at a low risk of bias, and 35 (48.6%) were at a high risk of bias. Thirty-eight (53%) of the trials were underpowered. We identified a strong correlation between a trial's FI or FQ and the trial's power.

Our study found that trials that provided strong evidence for orthopaedic surgery guidelines were largely fragile, underpowered, and at risk of bias.