Investigation of Missing Responses in Q-Matrix Validation

Area of Science:

• Psychometrics
• Educational Measurement
• Data Science

Background:

• Missing data poses challenges for Q-matrix validation in cognitive diagnostic models.
• Accurate Q-matrix validation is crucial for model implementation and interpretation.

Purpose of the Study:

• To investigate the impact of missing response handling strategies on Q-matrix validation methods.
• To compare the performance of the EM-based δ-method and the nonparametric Q-matrix refinement method under various missing data conditions.

Main Methods:

• Simulation study evaluating four missing data handling approaches: treat as incorrect, logistic regression, listwise deletion, and expectation-maximization (EM) imputation.
• Assessment of two Q-matrix validation methods: EM-based δ-method and nonparametric Q-matrix refinement.
• Analysis across multiple factors including missing rates, number of attributes, and number of items.

Main Results:

• EM imputation and logistic regression significantly improved Q-matrix validation performance compared to treating missing responses as incorrect or using listwise deletion.
• The nonparametric Q-matrix refinement method generally outperformed the EM-based δ-method.
• Higher missing rates consistently led to poorer performance for both validation methods.
• The number of attributes and items also influenced method performance.

Conclusions:

• Imputation methods like EM and logistic regression are recommended for handling missing data in Q-matrix validation.
• The nonparametric Q-matrix refinement method shows greater robustness in the presence of missing data.
• Researchers should carefully consider missing data handling strategies and their impact on Q-matrix validation results.