Detecting and describing heterogeneity in meta-analysis

Area of Science:

• Biostatistics
• Medical Research Methodology

Background:

• Investigating heterogeneity is essential in meta-analysis.
• The statistical power of heterogeneity tests is often stated as low but not well quantified.
• Normality assumptions in standard meta-analysis methods are frequently overlooked.

Purpose of the Study:

• To quantify the power of the heterogeneity test in meta-analysis.
• To examine how power is influenced by the number of studies, total information, and weight distribution.
• To introduce methods for assessing model conformity and investigating heterogeneity.

Main Methods:

• Simulations were used to assess the power of the heterogeneity test.
• The influence of the number of studies, total information (inverse variance), and weight distribution was investigated.
• Normal plots and associated tests were developed for model assessment.

Main Results:

• Meta-analysis power increases with total information, not solely the number of studies.
• Power is significantly reduced when one study dominates the total information.
• Normal plots provide a useful tool for assessing fixed-effect versus random-effects models.

Conclusions:

• The statistical test for heterogeneity should not be the only factor in choosing a meta-analysis model.
• Inspection of normal plots and clinical judgment are crucial for investigating and modeling heterogeneity.
• These findings have implications for the robust application of meta-analysis in medical research.