Novel item selection strategies for cognitive diagnostic computerized adaptive testing: A heuristic search framework

Area of Science:

• Psychometrics
• Educational Measurement
• Computerized Adaptive Testing

Background:

• Computerized adaptive diagnostic testing (CD-CAT) offers personalized measurements by tailoring items to individual ability.
• Effective item selection is critical for CD-CAT's accuracy and efficiency.
• Existing greedy strategies struggle to balance diagnostic performance and item bank utilization.

Purpose of the Study:

• To develop a novel item selection framework for CD-CAT using a graph-based heuristic search approach.
• To address the limitations of greedy item selection strategies in balancing diagnostic accuracy and item bank utilization.
• To enhance the efficiency and effectiveness of personalized measurements in cognitive diagnostic testing.

Main Methods:

• Transformed the CD-CAT item selection problem into a path-searching problem on a graph.
• Defined a heuristic function to evaluate the diagnostic potential of item selection paths.
• Proposed two search mechanisms with varying item exposure control to find optimal paths.
• Selected the first unused item from the identified optimal path for administration.

Main Results:

• The proposed heuristic search framework demonstrated enhanced item bank utilization compared to classic strategies.
• Diagnostic performance was maintained at a smaller cost, indicating improved efficiency.
• Simulation studies under various conditions validated the framework's effectiveness.

Conclusions:

• The novel CD-CAT item selection framework based on heuristic search offers a superior balance between diagnostic performance and item bank utilization.
• This approach represents a significant advancement in personalized measurement and cognitive diagnostic testing.
• The findings suggest improved strategies for adaptive testing item selection.