Stratified Item Selection Methods in Cognitive Diagnosis Computerized Adaptive Testing.

Area of Science:

• Educational Measurement
• Psychometrics
• Computerized Adaptive Testing

Background:

• Cognitive diagnostic computerized adaptive testing (CD-CAT) integrates cognitive diagnosis models (CDMs) with computerized adaptive testing (CAT) for enhanced diagnostic information.
• Effective item selection is crucial for CD-CAT's success in accurately classifying examinee proficiency.

Purpose of the Study:

• To propose and evaluate novel stratified item selection methods for CD-CAT.
• To compare the performance of these new methods against existing non-stratified approaches.

Main Methods:

• Developed stratified versions of posterior-weighted Kullback-Leibler (PWKL), nonparametric item selection (NPS), and weighted nonparametric item selection (WNPS) methods, termed S-PWKL, S-NPS, and S-WNPS.
• Utilized both original and novel stratification indices for performance evaluation.
• Conducted simulation studies manipulating sample size, item quality, attribute number, strata number, and data generation models.

Main Results:

• The S-WNPS and S-NPS methods demonstrated comparable and superior performance over S-PWKL.
• Item selection methods employing novel stratification indices slightly outperformed those using original indices.
• Non-stratified methods yielded the poorest performance in the simulations.

Conclusions:

• Stratified item selection methods, particularly S-WNPS and S-NPS, enhance the diagnostic accuracy of CD-CAT.
• Novel stratification indices offer a slight advantage over original ones.
• The findings underscore the importance of stratification in optimizing CD-CAT item selection for improved examinee classification.