A causal theory of error scores

Area of Science:

• Psychometrics
• Causal Inference
• Measurement Theory

Background:

• Modern test theory models response variables as functions of latent variables and unique error variables.
• Interpretation of error variables in psychometric models is often implicit, despite being central to many assumptions.
• Reasoning about psychometric models necessitates understanding the nature of error variables.

Purpose of the Study:

• To propose a causal theory of error to provide a framework for understanding error variables in psychometric models.
• To distinguish between item-specific causes of error: characteristic variables (person-varying) and circumstance variables (occasion-varying).
• To demonstrate how different assumptions about these error causes impact psychometric models and their implications.

Main Methods:

• Developed a causal framework for error in measurement.
• Differentiated between characteristic and circumstance variables as sources of item-specific error.
• Analyzed the implications of different error assumptions on psychometric models, probabilistic interpretations, and reliability.

Main Results:

• Assumptions about the unique causes of error imply different psychometric models.
• Different error assumptions have varied consequences for item bias, local homogeneity, reliability coefficient α, and test-retest correlation.
• A causal perspective on error variance aids in motivating modeling choices.

Conclusions:

• A causal theory of error enhances the understanding of data-generating mechanisms in psychometrics.
• Distinguishing between characteristic and circumstance variables offers a more nuanced approach to error analysis.
• This framework empowers researchers to make more informed decisions regarding psychometric model selection and interpretation.