Evaluating cognitive theory: a joint modeling approach using responses and response times.

Area of Science:

• Psychological assessment
• Cognitive psychology
• Educational measurement

Background:

• Current psychological research often overlooks response times in computer-based assessments, focusing solely on accuracy scores.
• Analyzing accuracy and response times separately can lead to confounded information and limited insights into cognitive processes.
• Existing methods struggle to simultaneously estimate person and task parameters from rule-based cognitive tests.

Purpose of the Study:

• To develop a novel model for the simultaneous analysis of accuracy scores and response times in rule-based cognitive tests.
• To enable the estimation of ability and speed (person parameters) alongside difficulty and time intensity (task parameters).
• To integrate task design parameters to assess their impact on task difficulty and time intensity, offering deeper insights into task structure.

Main Methods:

• A new statistical model is developed for the simultaneous analysis of accuracy and response time data.
• The model estimates person parameters (ability, speed) and task parameters (difficulty, time intensity).
• A Bayesian approach utilizing Markov Chain Monte Carlo (MCMC) methods is employed for model estimation.

Main Results:

• The model successfully dissociates confounded information often present in separate analyses of accuracy and response times.
• It allows for the simultaneous estimation of ability, speed, difficulty, and time intensity.
• Integration of design matrices enables assessment of cognitive processes' effects on task difficulty and time intensity.

Conclusions:

• The developed model offers a more integrated and informative approach to analyzing cognitive test data.
• It enhances the understanding of cognitive processes and task structures by utilizing both accuracy and response time information.
• The application in educational assessment demonstrates the model's utility in large-scale investigations of cognitive abilities like figural reasoning.