A Blocked-CAT Procedure for CD-CAT

Area of Science:

• Educational Measurement
• Psychometrics
• Computerized Adaptive Testing

Background:

• Traditional computerized adaptive testing (CAT) often lacks item review functionality.
• Cognitive Diagnosis Computerized Adaptive Testing (CD-CAT) aims to diagnose specific attributes.
• Item review can potentially improve test-taker performance and reduce anxiety.

Purpose of the Study:

• To introduce and evaluate a blocked-design procedure for CD-CAT that permits item review.
• To investigate the impact of various factors (item quality, block size, etc.) on classification accuracy.
• To assess the benefits of item review and its effect on test anxiety.

Main Methods:

• Development of four blocking versions for CD-CAT.
• Simulation studies to analyze classification rates under different conditions.
• Comparison of item selection indices within the new blocked-design framework.
• Examination of the influence of item review on attribute classification accuracy.

Main Results:

• The blocked-design CD-CAT procedure is promising, with item review causing only minor accuracy losses in certain scenarios.
• The procedure effectively alleviates the negative impact of test anxiety on examinee performance.
• Factors such as item quality and block size influence classification accuracy.

Conclusions:

• Blocked-design CD-CAT with item review is a viable approach for cognitive diagnosis.
• Allowing item review offers benefits in terms of test-taker experience and anxiety reduction.
• Further research can optimize blocking strategies and item selection for enhanced CD-CAT.