A Testlet Diagnostic Classification Model with Attribute Hierarchies

Area of Science:

• Psychometrics
• Educational Measurement
• Cognitive Diagnosis

Background:

• Cognitive Diagnostic Models (CDMs) are essential for understanding student mastery of specific skills.
• Existing CDMs often do not fully account for complex data structures like attribute hierarchies and item bundles (testlets).
• Accurate parameter estimation is crucial for the validity of diagnostic classifications.

Purpose of the Study:

• To introduce a Testlet Hierarchical Diagnostic Classification Model (TH-DCM) that integrates attribute hierarchies and testlet structures.
• To evaluate the parameter recovery of TH-DCM using a simulation study.
• To compare TH-DCM with the Testlet Higher-Order CDM (THO-DCM).

Main Methods:

• Development of the TH-DCM incorporating attribute hierarchies and testlet effects.
• Parameter estimation using the expectation-maximization algorithm with analytic dimension reduction.
• Simulation studies under various conditions to assess parameter recovery and model comparison.

Main Results:

• Ignoring significant testlet effects negatively impacts parameter recovery.
• CDMs with equal testlet effects performed comparably to those with unequal effects.
• Misspecifying the joint attribute distribution differentially affected parameter recovery.
• THO-DCM demonstrated robustness as an alternative to TH-DCM under certain hierarchical conditions.

Conclusions:

• The proposed TH-DCM offers a way to model complex attribute and item structures in cognitive diagnosis.
• Model misspecifications, particularly regarding testlet effects and attribute distributions, can bias parameter estimates.
• THO-DCM presents a viable and robust alternative for cognitive diagnosis when hierarchical structures are present.