Revision Q-matrix in middle school chemistry: a structural equation modeling approach

Area of Science:

• Educational Measurement
• Cognitive Psychology
• Chemistry Education

Background:

• Q-matrix accuracy is crucial for reliable cognitive diagnosis.
• Inaccurate Q-matrices are a common challenge in educational assessment research.
• This study addresses Q-matrix validation using a junior high chemistry topic.

Purpose of the Study:

• To validate and revise a Q-matrix for a cognitive diagnostic assessment tool.
• To compare the effectiveness of a Structural Equation Modeling (SEM) approach with other Q-matrix estimation methods.
• To improve the accuracy and reliability of cognitive diagnostic assessments.

Main Methods:

• Utilized Structural Equation Modeling (SEM) for Q-matrix validation and revision.
• Integrated qualitative disciplinary analysis with SEM fit indices (e.g., MI, model fit).
• Employed the generalized deterministic input, noisy "and" gate (GDINA) model for comparative analysis.

Main Results:

• The SEM-derived Q-matrix demonstrated excellent absolute fit and superior relative fit compared to benchmark methods.
• While absolute fit was slightly lower than a benchmark, relative fit indicated better overall structure.
• The SEM approach's qualitative focus influenced, but ultimately improved, Q-matrix performance.

Conclusions:

• Structural Equation Modeling offers a robust method for Q-matrix validation and revision in cognitive diagnosis.
• Integrating disciplinary knowledge with statistical analysis enhances Q-matrix quality.
• The SEM approach provides a valuable alternative for improving cognitive diagnostic assessment accuracy.