Related Experiment Video
Updated: Aug 31, 2025
13:00
Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments
Published on: January 23, 2017
10.0K
Application of Change Point Analysis of Response Time Data to Detect Test Speededness
1University of Notre Dame, Notre Dame, IN, USA.
Educational and Psychological Measurement
|August 22, 2022
Summary
This study introduces a change point analysis (CPA) method to identify test speededness using response time data. The new procedure effectively detects speededness with controlled false positives, enhancing computer-based testing analysis.
Area of Science:
- Educational Measurement
- Psychometrics
- Data Analysis
Background:
- Computer-based and web-based testing generate extensive response time data.
- Response time data is continuous and unobtrusively collected, offering advantages over traditional dichotomous or polytomous item response data.
- Accurate measurement is crucial in educational and psychological assessments.
Purpose of the Study:
- To propose and evaluate a change point analysis (CPA) procedure for detecting test speededness using response time data.
- To introduce likelihood ratio and Wald tests within the CPA framework for identifying speededness.
- To assess the performance of the CPA procedure in simulation studies.
Main Methods:
- Development of a change point analysis (CPA) procedure utilizing continuous response time data.
- Implementation of likelihood ratio and Wald test statistics for detecting abrupt changes indicative of speededness.
- Conducting simulation studies to evaluate the power and false positive rates of the CPA procedure with various critical values.
- Analysis of a real data example to demonstrate practical application and comparison with existing methods.
Main Results:
- The proposed CPA procedure demonstrates high statistical power in detecting test speededness.
- The false positive rate is maintained effectively under control, even with less stringent critical values.
- Accuracy in estimating the exact location of the change point is contingent upon the true change point's position.
- The CPA procedure shows utility in real-world data analysis, offering insights beyond response-only analyses.
Conclusions:
- Change point analysis provides a powerful tool for detecting test speededness using response time data in computer-based assessments.
- The proposed CPA method offers a reliable approach to identify potential issues with test-taking behavior, improving assessment validity.
- Further research can explore the nuances of change point estimation accuracy and its implications for interpreting assessment results.
