An empirical Q-matrix validation method for the sequential generalized DINA model.

Area of Science:

• Educational Measurement
• Psychometrics
• Cognitive Diagnosis

Background:

• The Q-matrix is crucial for cognitive diagnosis models but often subjectively created by experts.
• Misspecified Q-matrices can lead to inaccurate estimations of student attributes.
• Existing validation methods are primarily for dichotomous responses, limiting their application.

Purpose of the Study:

• To develop an empirical method for detecting and correcting Q-matrix misspecifications.
• To adapt Q-matrix validation for graded response data within the G-DINA model framework.
• To enhance the reliability of cognitive diagnosis models using graded response data.

Main Methods:

• Proposed a stepwise Q-matrix validation procedure.
• Utilized the sequential generalized deterministic inputs, noisy 'and' gate (G-DINA) model for graded responses.
• Employed the Wald test and an effect size measure for misspecification detection and correction.

Main Results:

• The proposed method effectively detects and corrects Q-matrix misspecifications in simulated graded response data.
• The feasibility and accuracy of the method were demonstrated through simulation studies.
• The approach was illustrated using real-world data from the TIMSS 2011 mathematics assessment.

Conclusions:

• The developed method provides a robust approach to empirically validate and correct Q-matrices for graded response data.
• This contributes to more accurate attribute estimation in cognitive diagnosis.
• The findings have implications for improving the quality of educational assessments and diagnostic feedback.