Assessing the properties of the prediction interval in random-effects meta-analysis

Area of Science:

• Biostatistics
• Medical Research Methodology
• Quantitative Synthesis

Background:

• Random-effects meta-analysis synthesizes findings across studies, estimating mean effects and summarizing heterogeneity.
• Prediction intervals are used to quantify heterogeneity, aiming to cover the true effect of future similar studies.

Purpose of the Study:

• To extensively investigate the performance of all published frequentist prediction interval methods in meta-analysis.
• To analyze the distribution of coverage probabilities and their dependence on heterogeneity and study numbers.

Main Methods:

• Mathematical background provided for prediction intervals.
• Extensive simulation study evaluating frequentist prediction interval methods.
• Focus on coverage probability distributions, interval lengths, and robustness to non-normality.

Main Results:

• Coverage probability distributions vary significantly with heterogeneity and the number of studies.
• For meta-analyses with few studies, coverage distributions are often asymmetric and unideal.
• Mean coverage alone can be misleading; distribution shape is critical for accurate interpretation.

Conclusions:

• Relying solely on average coverage for prediction intervals can lead to misinterpretation, particularly in small meta-analyses.
• The distribution of coverage probabilities is essential for a comprehensive understanding of prediction interval performance.
• Further investigation into interval length and robustness to effect non-normality is warranted.