Latent variables should remain as such: Evidence from a Monte Carlo study

Area of Science:

• Psychometrics
• Statistical Modeling

Background:

• Estimating relationships between latent variables using manifest scores can lead to attenuated (underestimated) results.
• This attenuation is known for Classical Test Theory (CTT) raw scores and factor analysis factor scores.
• Prior research has not fully explored this issue for Item Response Theory (IRT) theta estimates, despite their recommended use.

Purpose of the Study:

• To evaluate the impact of using subject scores as manifest variables in regression models for assessing latent variable relationships.
• To compare the accuracy of IRT theta estimates against Structural Equation Modeling (SEM) for estimating latent variable correlations.
• To provide recommendations for applied researchers regarding the best methods for estimating latent variable relationships.

Main Methods:

• A Monte Carlo simulation study was conducted.
• Key variables manipulated included sample size, number of items, test type, and the true correlation between latent variables.
• Methods compared included raw scores, three IRT theta estimation methods, and latent variable SEM.

Main Results:

• Estimates of the relationship between latent variables were consistently more accurate when using SEM.
• IRT models, while advantageous in other contexts, did not yield more accurate relationship estimates compared to SEM in this study.
• The accuracy of score-based estimates was found to be lower than SEM, regardless of the specific IRT method used.

Conclusions:

• Structural Equation Modeling (SEM) is the preferred method for accurately estimating relationships between latent variables.
• Applied researchers should consider using SEM over score-based methods (CTT, IRT) for this specific analytical purpose.
• The findings highlight the limitations of using derived scores as manifest variables in regression analyses for latent constructs.