Reducing the Misclassification Costs of Cognitive Diagnosis Computerized Adaptive Testing: Item Selection With...

Area of Science:

• Educational Measurement
• Psychometrics
• Cognitive Science

Background:

• Cognitive diagnosis computerized adaptive testing (CD-CAT) identifies examinee strengths/weaknesses on latent attributes.
• Misclassification costs vary, necessitating efficient item selection in CD-CAT.

Purpose of the Study:

• Propose and evaluate Minimum Expected Risk (MER) for item selection in CD-CAT.
• Incorporate varying misclassification costs to enhance measurement efficiency.

Main Methods:

• Developed MER based on Bayesian decision theory.
• Conducted simulations comparing MER with other item selection criteria (MPWKL, PWCDI, SHE, PWACDI).
• Evaluated classification accuracy and test efficiency for different attribute profiles.

Main Results:

• MER-U0 and MER-U1 demonstrated superior classification accuracy and efficiency for identifying no/full mastery profiles, especially with short tests or low-quality item banks.
• MER, MPWKL, PWCDI, and SHE showed similar performance for other profiles, outperforming PWACDI.
• MER, MER-U0, MER-U1, and PWACDI showed more effective item bank utilization.

Conclusions:

• MER is a feasible approach for CD-CAT item selection.
• MER enhances accuracy for specific attribute profiles, addressing diverse user needs.
• MER improves measurement efficiency in CD-CAT, particularly in challenging conditions.