Standardized mean differences cause funnel plot distortion in publication bias assessments.

Area of Science:

• Biomedical Research Synthesis
• Meta-Analysis Methodology
• Statistical Bias Detection

Background:

• Meta-analyses are crucial for synthesizing biomedical evidence.
• Publication bias assessment often relies on funnel plot asymmetry.
• Existing funnel plot methods may be susceptible to distortion.

Purpose of the Study:

• To investigate the influence of normalization, sample size, and intervention effects on funnel plot asymmetry.
• To evaluate the reliability of Standardized Mean Difference (SMD) versus standard error (SE) in funnel plots for publication bias detection.

Main Methods:

• Utilized empirical datasets and simulations to study funnel plot behavior.
• Assessed the impact of different normalization approaches and study characteristics.
• Compared Standardized Mean Difference (SMD) and Normalised Mean Difference (NMD) as effect measures.

Main Results:

• Funnel plots of SMD against SE are prone to distortion, overestimating publication bias.
• Distortion is exacerbated by small primary study sample sizes and the presence of intervention effects.
• Alternative methods, such as using NMD or plotting SMD against sample size, offer greater reliability.

Conclusions:

• Standard funnel plots using SMD and SE are unsuitable for accurate publication bias assessment.
• These methods can lead to false-positive conclusions regarding publication bias.
• Recommended alternative plotting strategies enhance the reliability of bias detection in meta-analyses.