Unveiling local dependencies in accuracy and speed: A mixture hierarchical modeling approach

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Hierarchical models (HMs) are standard for analyzing response accuracies and response times (RTs).
• Population-level correlations between ability and speed may not capture local dependencies, potentially invalidating inferences.
• Existing models struggle with heterogeneous response speeds and item-level dependencies.

Purpose of the Study:

• To propose a novel mixture-based hierarchical model (Mix-HM) addressing limitations of conventional HMs.
• To investigate the impact of latent speed heterogeneity and local dependencies on parameter estimation.
• To evaluate the model selection capabilities of Bayesian information criteria.

Main Methods:

• Development of a mixture-based hierarchical model (Mix-HM) incorporating latent speed heterogeneity and item-level dependencies.
• Two simulation studies: one for parameter recovery and one for model selection using Bayesian information criteria.
• Empirical analysis using large-scale assessment data to compare Mix-HM with conventional HMs.

Main Results:

• The Mix-HM demonstrated satisfactory parameter recovery, outperforming conventional HMs which showed biased estimates under local dependencies.
• Bayesian information-based criteria generally succeeded in identifying the correct model across various conditions.
• The Mix-HM provided better model fit and more stable parameter estimates in empirical data, unlike conventional HMs which distorted ability-speed relationships.

Conclusions:

• Accounting for latent speed heterogeneity and local dependencies is crucial for valid inferences when using RTs in assessments.
• The proposed Mix-HM offers a more robust approach for jointly modeling accuracy and RT data.
• This research underscores the importance of advanced modeling techniques for accurate measurement in educational and psychological assessments.