Confidence intervals for population reliability coefficients: Evaluation of methods, recommendations, and software...

Area of Science:

• Psychometrics
• Statistical modeling
• Measurement theory

Background:

• Composite scores are widely used across disciplines.
• Estimating the reliability of these scores is essential for accurate interpretation.
• Point estimates of reliability do not convey uncertainty, necessitating confidence intervals.

Purpose of the Study:

• To evaluate various confidence interval methods for four common reliability coefficients.
• To determine the most appropriate confidence interval methods under different conditions.
• To provide practical recommendations for reliability estimation in psychometric research.

Main Methods:

• Conducted three large-scale Monte Carlo simulation studies.
• Evaluated confidence interval methods for coefficient alpha, coefficient omega, hierarchical omega, and categorical omega.
• Assessed performance under a variety of simulated conditions.

Main Results:

• Bootstrap confidence intervals are generally recommended for hierarchical omega with continuous data.
• Bootstrap confidence intervals are generally recommended for categorical omega with categorical data.
• Findings provide guidance on selecting appropriate confidence intervals for reliability coefficients.

Conclusions:

• Bootstrap confidence intervals offer a reliable way to express uncertainty in reliability estimates.
• Specific recommendations are made based on item type (continuous vs. categorical).
• The discussed methods are available in the R package MBESS.