Confidence intervals for intraclass correlation coefficients in variance components models.

Area of Science:

• Statistics
• Biostatistics
• Psychometrics

Background:

• Calculating confidence intervals for intraclass correlation coefficients (ICC) in agreement studies is challenging due to model-specific limitations.
• Existing methods lack a generic approach applicable across different agreement study designs and variance component models.

Purpose of the Study:

• To develop and evaluate two novel, generic methods for constructing confidence intervals for intraclass correlation coefficients.
• To compare the performance of these new methods against existing model-specific approaches in various agreement study scenarios.

Main Methods:

• Two generic approaches for ICC confidence intervals were developed: one using Satterthwaite's approximation with an F-distribution, and another using moments with a Beta distribution.
• Both methods rely on restricted maximum likelihood (REML) estimates for variance components.
• Simulation studies were performed using one-way random effects and three-way variance component models, including balanced and unbalanced designs with small sample sizes.

Main Results:

• The F-distribution approach yielded acceptable coverage probabilities.
• The Beta distribution approach demonstrated accurate coverage probabilities in most investigated settings, outperforming the F-distribution approach, particularly in unbalanced designs.
• Both proposed methods were compared to existing model-specific confidence interval approaches.

Conclusions:

• The Beta distribution-based method provides a robust and accurate generic approach for confidence intervals of intraclass correlation coefficients in agreement studies.
• The developed methods offer improved statistical reliability for assessing agreement, especially in complex or unbalanced study designs.