Ignoring a Multilevel Structure in Mixture Item Response Models: Impact on Parameter Recovery and Model Selection

Area of Science:

• Psychometrics
• Statistical Modeling
• Educational Measurement

Background:

• Item response theory (IRT) models are crucial for educational and psychological assessments.
• Mixture IRT models account for unobserved heterogeneity within populations.
• Multilevel data structures, common in educational settings, require specialized modeling approaches.

Purpose of the Study:

• To investigate the consequences of ignoring multilevel structures in mixture item response models.
• To determine when a multilevel mixture item response model is necessary.
• To evaluate the impact on model selection and parameter recovery.

Main Methods:

• Two simulation studies were conducted.
• Study 1 examined ignoring multilevel dependency in latent classes.
• Study 2 assessed fitting a multilevel model to single-level data.

Main Results:

• Ignoring multilevel structures led to less accurate item discrimination parameter estimates and standard errors when the number of clusters exceeded 24 and cluster size exceeded six.
• The Bayesian Information Criterion (BIC) favored the multilevel model when cluster size and number of clusters were at least 50.
• Fitting a multilevel model to single-level data did not distort parameter estimates when cluster size was at least 20.

Conclusions:

• A multilevel mixture item response model is recommended for clustered data with specific cluster size and number of clusters.
• Ignoring multilevel structures can bias parameter recovery in mixture IRT models.
• The findings provide guidance on selecting appropriate IRT models for hierarchical data.