Assessing multiple abilities through process data in computer-based assessments: The multidimensional sequential...

Area of Science:

• Educational Measurement and Assessment
• Psychometrics
• Cognitive Psychology

Background:

• Computer-based assessment (CBA) increasingly utilizes process data to understand examinee response patterns.
• Traditional methods often focus on single ability estimation, limiting comprehensive assessment.
• There is a need for advanced models to leverage detailed process data for multidimensional ability assessment.

Purpose of the Study:

• To introduce the Multidimensional Sequential Response Model (MSRM) for analyzing process data in CBA.
• To develop and evaluate a Bayesian estimation method for the MSRM.
• To demonstrate the practical application and utility of the MSRM in educational and psychological contexts.

Main Methods:

• Development of the Multidimensional Sequential Response Model (MSRM).
• Application of a Bayesian estimation technique for MSRM parameters.
• Validation through a Monte Carlo simulation study with varying conditions (sample size, sequence length).
• Empirical testing using two real-world case studies.

Main Results:

• The MSRM demonstrated adequate accuracy and computational efficiency in parameter estimation.
• Estimation quality improved with larger sample sizes and longer response sequences.
• The model proved effective in practical application across different assessment contexts.
• The MSRM provided detailed insights into multidimensional ability mastery.

Conclusions:

• The MSRM is a viable and effective tool for assessing multiple latent abilities using process data from CBA.
• The proposed Bayesian estimation method is accurate and computationally efficient.
• The MSRM offers a pathway to more precise ability profiling, supporting tailored instruction and targeted interventions.