Standardization and other approaches to meta-analyze differences in means

Area of Science:

• Biostatistics
• Medical Research Methodology

Background:

• Meta-analysts frequently use standardized mean differences (SMD) to synthesize findings from studies employing diverse measurement instruments.
• Inappropriate calculation of the standardizing denominator (standard deviation) is common in meta-analyses, particularly when combining controlled trials and crossover designs.

Purpose of the Study:

• To demonstrate the correct calculation of standardized mean differences (SMD).
• To highlight the biases introduced by common, incorrect methods of calculating SMDs.
• To present alternative, more appropriate methods for meta-analyzing effects from disparate measures.

Main Methods:

• The tutorial details the correct calculation of SMD using a pre-intervention standard deviation.
• It critiques common but inappropriate standardizing denominators (pooled SD of change or post-intervention scores).
• Alternative meta-analysis techniques are discussed, including log transformation of response ratios and rescaling methods.

Main Results:

• Incorrectly calculated SMDs are biased and difficult to interpret, often due to misleading guidance in publications and software.
• Even with correct standardization, heterogeneity can be artifactually increased by variations in the standardizing SD across settings.
• Existing magnitude thresholds for SMDs are not indicative of clinically important differences.

Conclusions:

• Accurate SMD calculation is crucial for valid meta-analysis; using pre-intervention SD is recommended.
• Alternative methods like log transformation and rescaling are necessary for combining disparate measures effectively.
• Clinically meaningful interpretation requires methods beyond standard SMD thresholds, such as rescaling with minimum clinically important differences.