Interactions Between Termination Criteria and Ability Estimators in Computerized Adaptive Testing

Area of Science:

• Psychometrics
• Educational Measurement
• Computerized Adaptive Testing (CAT)

Background:

• Computerized adaptive testing (CAT) optimizes measurement by tailoring items to examinees.
• The effectiveness of CAT relies on ability estimators and termination criteria.
• Limited research exists on the interaction of these components across diverse item banks.

Purpose of the Study:

• To investigate the interactive effects of ability estimators and termination criteria on CAT performance.
• To evaluate these interactions across varying item bank sizes and information distributions.
• To identify optimal CAT configurations for different item bank characteristics.

Main Methods:

• Simulation study evaluating four ability estimators (MLE, WLE, MAP, EAP).
• Assessed four termination criteria (fixed-length, SEM, MI, Δθ).
• Tested across low- (100-item) and high- (500-item) information banks with flat and peaked distributions using the three-parameter logistic model.

Main Results:

• Optimal CAT configuration depends on item bank size and shape.
• WLE proved the most robust estimator, mitigating MLE boundary issues and Bayesian shrinkage bias.
• In high-information banks, SEM and fixed-length rules minimized bias and RMSE.
• In low-information peaked banks, the Δθ rule with WLE balanced accuracy and efficiency, avoiding inefficient test elongation.

Conclusions:

• No single CAT design fits all scenarios; optimization is context-dependent.
• For high-quality item banks, WLE with SEM or fixed-length rules is recommended.
• For lower-quality banks, Δθ or hybrid SEM rules are advised to prevent inefficient test length.