Modeling Not-Reached Items in Timed Tests: A Response Time Censoring Approach

Area of Science:

• Educational Measurement
• Psychometrics
• Statistics

Background:

• Timed computer-based assessments often result in missing data due to examinees not reaching all items.
• Understanding the reasons for missing data is crucial for accurate assessment results.

Purpose of the Study:

• To propose and evaluate a novel statistical approach for accounting for missing not-reached items in timed assessments.
• To incorporate response time censoring directly into the statistical model for item responses.

Main Methods:

• Developed a marginal maximum likelihood estimator that integrates response time censoring into the observed likelihood function.
• Established the asymptotic properties of the proposed estimator.
• Conducted simulation studies to compare the new method with alternatives that ignore censoring.

Main Results:

• The proposed method, which accounts for response time censoring, demonstrated superior performance compared to methods ignoring this mechanism in simulation studies.
• Empirical validation using data from the PISA 2018 Science Test supported the effectiveness of the approach.

Conclusions:

• Response time censoring is a critical factor to consider when analyzing data from timed computer-based assessments.
• The proposed method provides a statistically sound framework for handling not-reached items, leading to more accurate inferences.