Related Experiment Videos
Real-Time Protection of Person-Parameter Updating and Item Selection in Computerized Adaptive Testing Under Item
Chunyu Piao1, Jiwei Zhang1, Jing Lu1
1Northeast Normal University, Changchun, Jilin, China.
Abstract:
Computerized adaptive testing (CAT) is designed to enhance measurement precision and testing efficiency by sequentially updating interim person-parameter estimates and using these estimates to guide subsequent item selection and test termination. However, responses affected by item preknowledge can distort these estimates and, in turn, alter the subsequent course of the adaptive test. We propose a response-time-informed real-time adaptive protection framework that integrates within-examinee monitoring, protected person-parameter updating, and adaptive item selection within the CAT loop. Response time is monitored item by item as a dynamic behavioral indicator using a sequential Bayesian changepoint detection algorithm. When posterior evidence suggests an abrupt within-test change, the response-time signal is combined with response correctness to determine whether the current response is potentially affected by item preknowledge. Flagged responses are excluded from interim ability and speed updating, and the most recent protected estimates guide subsequent item selection and stopping decisions; otherwise, CAT proceeds according to the standard updating rule. We evaluate the framework under alternative item-selection criteria, varying prevalence and severity of item preknowledge, four item time-intensity distributions, and fixed-length and precision-based stopping rules. Simulation results show that the proposed framework achieves strong detection performance at both the examinee and response levels, limits the propagation of contaminated responses through the adaptive loop, and improves the accuracy and precision of person-parameter estimation, particularly when item preknowledge is more prevalent or severe.