Compromised Item Detection for Computerized Adaptive Testing.

Area of Science:

• Psychometrics
• Educational Measurement
• Computerized Adaptive Testing (CAT)

Background:

• Item leakage compromises the integrity of computer-based tests, particularly CAT.
• Existing detection methods are often computationally intensive, hindering real-time application.
• Previous sequential methods for detecting compromised items did not account for diverse leakage scenarios.

Purpose of the Study:

• To introduce a novel model with a leakage parameter for characterizing item leaking processes.
• To develop a generalized detection method for compromised items in CAT.
• To improve the accuracy and efficiency of real-time monitoring for item leakage.

Main Methods:

• Development of a new statistical model incorporating a leakage parameter.
• Application of the model to detect compromised items across various leakage speeds (fast and slow).
• Validation of the detection method's accuracy and type-I error control.

Main Results:

• The proposed model accurately detects compromised items in both fast and slow leakage scenarios.
• The method maintains the type-I error rate at the nominal level.
• The model successfully estimates the time point of item compromise.

Conclusions:

• The new generalized detection method offers high accuracy and efficiency for identifying compromised items in CAT.
• The model's ability to estimate the time of compromise provides valuable insights for test administrators.
• This approach enhances the fairness and validity of computer-based testing by addressing item leakage effectively.