Has the Time Come to Stop Using the "Standardised Mean Difference"?

Area of Science:

• Biostatistics
• Medical Research Methodology

Background:

• Meta-analyses commonly employ the standardized mean difference (SMD) to synthesize study outcomes.
• The SMD, while prevalent, possesses inherent limitations impacting its interpretation and application.

Purpose of the Study:

• To critically examine the limitations of the standardized mean difference in meta-analyses.
• To propose alternative approaches for enhancing the clarity and clinical relevance of effect size reporting.

Main Methods:

• A review of the conceptual and practical challenges associated with the SMD was conducted.
• Alternative effect size metrics, including the binomial effect size display and number-needed-to-treat, were explored.
• The utility of binary outcomes for continuous data was considered.

Main Results:

• Key limitations of the SMD include its abstract statistical nature, potential misinterpretation of clinical significance, unmet statistical assumptions, and difficulty in communicating findings to non-researchers.
• The binomial effect size display and number-needed-to-treat offer more intuitive interpretations.
• Binary outcomes may enhance understandability, though selecting the optimal binary outcome for continuous data remains challenging.

Conclusions:

• The standardized mean difference remains a valuable tool in meta-analysis when its limitations are acknowledged.
• Incorporating binary outcomes alongside SMD can improve the accessibility and interpretability of research findings.