A mixture model for responses and response times with a higher-order ability structure to detect rapid guessing...

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Multidimensional latent traits with hierarchical structures are common in assessments.
• Test-taking behaviors, including rapid guessing, impact data validity.
• Existing models may struggle to accurately detect guessing in complex assessment structures.

Purpose of the Study:

• To propose a novel mixture model for response and response time data.
• To incorporate hierarchical ability structures and auxiliary information for improved guessing detection.
• To enhance the accuracy and efficiency of identifying rapid guessing behavior in assessments.

Main Methods:

• Development of a mixture model integrating response and response time data.
• Inclusion of hierarchical ability structures and inter-subtest correlations.
• Application of Markov chain Monte Carlo (MCMC) for model parameter estimation.
• Utilizing deviance information criterion (DIC) and log pseudo-marginal likelihood (LPML) for model fit evaluation.

Main Results:

• The proposed model effectively recovers parameters with reduced bias and mean squared error compared to unidimensional models.
• Demonstrated higher true positive rates and significantly lower false detection rates for rapid guessing.
• Simulation studies confirmed the model's robustness and accuracy.
• Real data analysis validated the practical utility of the model.

Conclusions:

• The new hierarchical mixture model offers a superior approach for detecting rapid guessing behavior in complex assessments.
• Improved detection of guessing enhances the validity and fairness of educational and psychological testing.
• The model provides a valuable tool for researchers and practitioners in psychometrics and educational measurement.