Combining regularization and logistic regression model to validate the Q-matrix for cognitive diagnosis model

Area of Science:

• Educational Measurement
• Psychometrics
• Data Science

Background:

• Q-matrices are crucial for cognitive diagnosis models (CDMs) but often rely on subjective expert judgment, risking errors.
• Existing statistical validation methods for Q-matrices, like MLR-B and Hull, have limitations in time or accuracy.
• Misspecified Q-matrices can negatively impact the reliability of diagnostic assessments.

Purpose of the Study:

• To develop and evaluate a novel statistical method for Q-matrix validation.
• To improve the accuracy and efficiency of Q-matrix validation in cognitive diagnosis.
• To address the limitations of existing Q-matrix validation techniques.

Main Methods:

• A new method combining L1 regularization with the multiple logistic regression-based (MLR-B) model was proposed.
• An L1 penalty term was applied to the MLR model's log-likelihood to refine attribute selection for each item.
• A simulation study compared the regularized MLR-B method against the traditional MLR-B and Hull methods.

Main Results:

• The regularized MLR-B method demonstrated superior Q-matrix recovery rate (QRR) and true positive rate (TPR), particularly with small sample sizes.
• The new method achieved a slightly higher true negative rate (TNR) compared to existing methods.
• Computational efficiency was improved, with the regularized method requiring less time than MLR-B and comparable time to the Hull method.

Conclusions:

• The regularized MLR-B method offers a more accurate and efficient approach to Q-matrix validation in cognitive diagnosis.
• This method effectively addresses the issue of misspecified Q-matrices, enhancing the reliability of CDMs.
• The findings suggest practical advantages for using this regularized approach in real-world assessment applications.