Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing.

Area of Science:

• Psychometrics
• Educational Measurement
• Computerized Adaptive Testing (CAT)

Background:

• Item compromise remains a significant threat to the integrity of computerized adaptive testing (CAT).
• Existing methods for detecting compromised items primarily rely on monitoring the proportion of correct responses.
• Response times offer valuable, yet underutilized, data for identifying leaked test items.

Purpose of the Study:

• To propose and evaluate augmented methods for detecting compromised items in CAT.
• To investigate the utility of simultaneously monitoring response accuracy and response times.
• To improve the detection power and reduce false positives in identifying compromised test items.

Main Methods:

• Development of several augmented methods for compromised item detection.
• Simultaneous monitoring of changes in proportion correct and average response time for each item.
• Application of various moving average strategies to track item performance in real time.

Main Results:

• Augmented methods incorporating response times demonstrated marked improvements in detection power.
• The combined analysis of response accuracy and response times led to a reduction in false positives.
• Simulation results with an operational item pool validated the effectiveness of the proposed methods.

Conclusions:

• Integrating response time data significantly enhances the detection of compromised items in CAT.
• The proposed methods offer a more robust approach to maintaining test security.
• Future research should explore further refinements of these dual-monitoring strategies.