A multiple-choice SDT model for cognitive diagnosis models

Area of Science:

• Educational Measurement
• Psychometrics
• Cognitive Psychology

Background:

• Multiple-choice (MC) item analysis often employs signal detection theory (SDT) to model examinee response processes.
• Cognitive diagnosis models (CDMs) aim to identify specific skills or knowledge mastered by examinees.
• Existing frameworks may lack a unified approach to understanding examinee choices within CDMs.

Purpose of the Study:

• To integrate the multiple-choice signal detection theory (MC-SDT) model with various cognitive diagnosis models (CDMs).
• To establish a unified theoretical framework for analyzing examinee behavior on MC items within CDMs.
• To demonstrate the parsimony and utility of the proposed MC-SDT-CDM models.

Main Methods:

• The study conceptualizes both MC-SDT and CDMs as hierarchical models with item-level (Level 1) and examinee-level (Level 2) components.
• The MC-SDT model is proposed as the Level 1 model within a broader CDM framework (Level 2).
• The methodology involves demonstrating how MC-SDT-CDM models are parsimonious sub-models of existing MC-CDMs.

Main Results:

• The integration results in MC-SDT-CDM models that unify response processes across different CDMs.
• These models offer straightforward interpretations at the MC-SDT (Level 1) level.
• The proposed models avoid common estimation problems and are suitable for small sample sizes.

Conclusions:

• The MC-SDT-CDM framework provides a unified and parsimonious approach to cognitive diagnosis for multiple-choice items.
• The models offer clear interpretations and practical advantages, particularly for datasets with limited sample sizes.
• The application to TIMSS 2007 data illustrates the practical utility of this integrated approach in educational measurement.