Online Item Calibration for Q-Matrix in CD-CAT

Area of Science:

• Educational Measurement
• Psychometrics
• Item Response Theory

Background:

• Maintaining large-scale item banks is crucial for educational and psychological assessments.
• Accurate item calibration is essential for reliable and valid test scores.
• Existing calibration methods may not fully address the complexities of modern testing formats.

Purpose of the Study:

• To propose and evaluate two novel methods for calibrating new test items.
• To adapt calibration procedures for the deterministic inputs, noisy-and-gate (DINA) model.
• To ensure the applicability of these methods to both fixed and adaptive item selection processes.

Main Methods:

• Utilizing maximum likelihood and Bayesian estimators for latent knowledge states.
• Applying calibration methods under the DINA model, incorporating the -matrix, slipping, and guessing parameters.
• Conducting extensive simulations to compare the performance of proposed methods.

Main Results:

• The proposed methods demonstrate effective item calibration for large-scale item banks.
• Both methods show robust performance across simulated paper-pencil and computerized adaptive testing scenarios.
• The study provides a comparative analysis of the proposed calibration approaches.

Conclusions:

• The developed calibration methods offer a valuable contribution to item bank maintenance and test development.
• These methods are flexible and can be extended to other diagnostic classification models.
• The findings support the efficient and accurate replenishment of item banks in various testing contexts.