Beyond the Usual Significance: Fragility Indices of Randomized Controlled Trials in Top General Orthopaedic Journals
Mohammad Poursalehian1, Mahdi Sahebi, Mahboobeh Tajvidi
1From the Joint Reconstruction Research Center, Tehran University of Medical Sciences, Tehran, Iran (Poursalehian, Sahebi, Tajvidi, and Sabaghian), the Students' Scientific Research Center (SSRC), Tehran University of Medical Sciences, Tehran, Iran (Sahebi), the Student Research Committee, Faculty of Medicine, Abadan University of Medical Sciences, Abadan, Iran (Tajvidi), the School of Medicine, Iran University of Medical sciences, Tehran, Iran (Sabaghian), the Orthopedic Surgery Research Center (OSRC), Sina University Hospital, Tehran University of Medical Sciences, Tehran, Iran (Asgari), the Department of Orthopaedic Surgery, Nationwide Children's Hospital, Columbus, OH (Tabaie), the Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada (Bhandari), the Division of Orthopedic Surgery, Department of Surgery, McMaster University, Hamilton, Ontario, Canada (Bhandari), and the International Center for Limb Lengthening, Rubin Institute for Advanced Orthopedics, Sinai Hospital of Baltimore, Baltimore, MD (Hoveidaei).
Introduction:
Understanding the reliability of outcomes in randomized controlled trials (RCTs) is crucial, as standard metrics, such as P- value do not fully capture result fragility. This led to the adoption of specific indices: the fragility index (FI), which measures the strength of trial through significant results by calculating the minimum number of patient status changes from nonevent to event required to make the results statistically insignificant; reverse fragility index (RFI), used for insignificant results; and continuous fragility index (CFI), which acts similar to FI for significant continuous outcomes. The objective is to assess the robustness of orthopaedics RCTs using these indices across leading orthopaedic journals.
Methods:
A systematic review of RCTs published between 2019 and 2023 in the top five general orthopaedic journals, identified through Scientific Journal Rankings, was done. Data extraction focused on FI, RFI, and CFI and related measures for 160 RCTs. The indices were calculated using established methodologies, with sample size adjustments.
Results:
22 RCTs had statistically significant dichotomous primary outcomes and 17 studies had notable dichotomous secondary outcomes. Twenty-nine had negligible (insignificant) dichotomous primary outcomes, and 92 reported notable continuous outcomes. Only one RCT reported a FI in the article. The median FI was 5 with a median sample size of 142 for dichotomous outcomes. The median RFI was 3 with a median sample size of 100 for negligible outcomes, and the median CFI was 13 with a median sample size of 86.5 for continuous outcomes, showing dichotomous outcomes to be more fragile than continuous ones.
Discussion:
Continuous outcomes are less fragile than dichotomous outcomes, with negligible dichotomous outcomes being particularly more fragile. This fragility stems from small sample sizes and limited outcome events. Using these indices, especially when considering patient loss to follow-up, can improve the reliability of findings.
Level Of Evidence:
I.
More Related Videos
Related Concept Videos
01:13Multiple Comparison Tests
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
01:10Bonferroni Test
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
01:09Comparing Experimental Results: Student's t-Test
01:12Non-destructive Tests for Concrete Strength
01:16Tensile Strength Considerations of Concrete
The dimensions and shape of a concrete specimen also...
01:17Cochran's Q Test