Meta-analytic pooling of intraclass correlation coefficient estimates

Area of Science:

• Statistics
• Biostatistics
• Psychometrics

Background:

• Intraclass correlation coefficient (ICC) estimates are crucial for statistical methods involving clustered data.
• Accurate ICC values are needed for power analyses, meta-analytic effect size adjustments, and reliability measures.
• ICC estimates are also used as primary outcomes in meta-analyses to pool agreement or reliability.

Purpose of the Study:

• To evaluate the accuracy of meta-analytically pooled ICC estimates.
• To compare different ICC variance formulas used as inverse variance weights in meta-analysis.
• To identify the best performing ICC variance formula for recovering population ICC parameters.

Main Methods:

• Meta-analysis simulation study.
• Evaluation of pooled ICC estimates derived from various ICC variance formulas.
• Comparison of recovered ICC values against population ICC parameters under different conditions.

Main Results:

• The variance formula employing a normalizing transformation demonstrated superior performance.
• This formula consistently recovered population ICC parameter values more accurately across most simulated conditions.
• Other ICC variance formulas showed varying degrees of bias and inaccuracy.

Conclusions:

• The choice of ICC variance formula significantly impacts the accuracy of meta-analytically pooled ICC estimates.
• Using the normalizing transformation variance formula is recommended for more reliable meta-analytic pooling of ICCs.
• Accurate ICC estimation is essential for valid secondary analyses and meta-analyses focusing on agreement or reliability.