Understanding measurement precision from a regression perspective

Area of Science:

• Psychometrics
• Statistical Modeling
• Measurement Theory

Background:

• Existing frameworks for measurement precision often focus on either reliability or prediction separately.
• McDonald's (2011) regression framework provides a foundation for unifying these perspectives.
• Integrating reliability (observed score accuracy) and prediction (latent score estimation) is crucial for comprehensive measurement evaluation.

Purpose of the Study:

• To extend McDonald's (2011) regression framework for measurement precision.
• To integrate the concepts of reliability and proportional reduction in mean squared error (PRMSE).
• To introduce and validate a Monte Carlo (MC) method for estimating these precision metrics.

Main Methods:

• Adopting and expanding a regression framework for measurement precision.
• Decomposing observed scores into true scores and error for reliability.
• Decomposing latent scores into optimal predictors (EAP scores) and prediction error for PRMSE.
• Utilizing Monte Carlo simulations for estimation when analytic solutions are complex or unavailable.

Main Results:

• Reliability and PRMSE are demonstrated as coefficients of determination in isomorphic regressions.
• The Monte Carlo method is shown to be a viable approach for estimating reliability and PRMSE.
• Illustrations provided for factor analysis, two-parameter logistic models, and a two-dimensional item response tree model.

Conclusions:

• The expanded regression framework offers a unified approach to measurement precision.
• The Monte Carlo method provides a flexible tool for estimating reliability and PRMSE across various statistical models.
• This work enhances the understanding and application of measurement precision in complex psychometric and statistical contexts.