When the p Value Doesn't Cut It: The Fragility Index Applied to Randomized Controlled Trials in Colorectal Surgery

Area of Science:

• Colorectal Surgery
• Clinical Trials
• Statistical Analysis

Background:

• The American Statistical Association recommends statistical methods beyond p ≤ 0.05.
• The fragility index measures the minimum patient outcome changes needed to alter statistical significance.
• Previous reviews indicate many randomized controlled trials (RCTs) have low fragility indices, suggesting limited robustness.

Purpose of the Study:

• To review the fragility indices of RCTs in colorectal surgery.
• To assess the robustness of published colorectal surgery RCTs.

Main Methods:

• A systematic PubMed search identified colorectal surgery RCTs published between 2016 and 2018 with dichotomous primary outcomes (p ≤ 0.05).
• Fragility index and number of patients lost to follow-up were the main outcome measures.
• The percentage of trials with lost patients exceeding the fragility index was calculated.

Main Results:

• 90 colorectal surgery RCTs met the inclusion criteria.
• The median fragility index was 3 (IQR, 1-10).
• In 57% of trials (51/90), the number of patients lost to follow-up was greater than the fragility index.

Conclusions:

• Most colorectal surgery RCTs exhibit low fragility indices.
• A significant proportion of trials (57%) had more lost patients than the fragility index, potentially altering study conclusions.
• Assessing clinical trial robustness beyond p-values is crucial for reliable clinical application.