Related Experiment Video
Updated: Sep 24, 2025
09:27
Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language
Published on: October 13, 2018
10.2K
Assessing the validity of test scores using response process data from an eye-tracking study: a new approach.
Victoria Yaneva1, Brian E Clauser2, Amy Morales2
1National Board of Medical Examiners, 3750 Market Street, Philadelphia, PA, 19104-3102, USA. vyaneva@nbme.org.
Summary
This study uses eye-tracking and machine learning to analyze how students answer multiple-choice questions (MCQs). Correct answers involve careful reading and stem-to-option processing, unlike incorrect ones.
Area of Science:
- Educational Measurement
- Cognitive Psychology
- Data Science
Background:
- Understanding test-taker response processes is crucial for validating score interpretations in multiple-choice questions (MCQs).
- Eye-tracking technology offers a promising method for capturing detailed data on cognitive processes during test-taking.
- Previous research has explored eye-tracking but lacked machine learning integration for score interpretation validity.
Purpose of the Study:
- To introduce a novel methodology for evaluating score interpretation validity using eye-tracking data and machine learning.
- To identify distinct eye-movement patterns associated with correct and incorrect responses in clinical MCQs.
- To assess the contribution of various eye-tracking features to the classification of response accuracy.
Main Methods:
- Collected eye-tracking data from 26 students answering clinical MCQs.
- Engineered 119 eye-tracking features as input for a machine learning model.
- Utilized machine learning to classify responses as correct or incorrect based on eye-movement patterns.
- Evaluated feature combinations to understand their predictive power.
Main Results:
- Eye-movement patterns differed significantly between correct and incorrect responses.
- Incorrect responses were linked to processing from options to the question stem.
- Correct responses were associated with stem-to-option processing, careful reading, and decisive option selection.
- Machine learning models demonstrated the ability to predict response accuracy from eye-tracking data.
Conclusions:
- The study validates a new data-driven approach to assessing score interpretation validity using eye-tracking and machine learning.
- Observed eye-movement behaviors for correct responses align with established theoretical models of test-taking.
- Identified response strategies associated with incorrect answers suggest potential areas for targeted remediation.
- This research pioneers the use of machine learning with eye-tracking for evaluating the validity of MCQ score interpretations.
