Disparity between statistical significance and clinical importance in published randomised controlled trials: a...

Area of Science:

• Clinical Trials Methodology
• Evidence-Based Medicine
• Biostatistics

Background:

• Interpreting randomized controlled trials (RCTs) requires evaluating both statistical significance and clinical importance.
• A potential disparity exists between these two measures, impacting the reliability of research findings.

Purpose of the Study:

• To estimate the extent of disparity between statistical significance and clinical importance in published RCTs.
• To identify factors associated with this statistical-clinical importance disparity.

Main Methods:

• A methodological study analyzed 500 two-arm, phase three, superiority RCTs in human health interventions published between 2018-2022.
• Clinical importance was determined by comparing the specified delta value or minimum clinically important difference with the effect size.
• Multinomial logistic regression explored factors associated with statistical significance and clinical importance disparities (SS+CI- and SS-CI+).

Main Results:

• Information to classify clinical importance was unavailable for 38.4% of trials.
• Overall disparity was observed in 20.5% of the remaining 308 trials.
• Statistically significant but not clinically important (SS+CI-) disparity occurred in 10.3% of trials, while statistically non-significant but clinically important (SS-CI+) disparity occurred in 29.5%.

Conclusions:

• Up to 20% of RCTs may exhibit a disparity between statistical significance and clinical importance.
• Emphasizes the need to consider clinical importance alongside statistical significance in trial interpretation.
• Recommends stringent adherence to reporting guidelines for improved clarity and reliability of RCTs.