Related Experiment Video
Updated: May 28, 2026
09:00
Advancing Dyslexia Assessment in Children Through Computerized Testing
Published on: August 16, 2024
Applying GenAI to Optimize Q-Matrix Construction for Cognitive Diagnostic Assessment in EFL Reading.
Wenbo Du1, Jiayi Shen1, Xiaomei Ma1
1School of Foreign Studies, Xi'an Jiaotong University, Xi'an 710049, China.
Journal of Intelligence
|May 26, 2026
Summary
Generative artificial intelligence (GenAI) can optimize Q-matrix construction for cognitive diagnostic assessment (CDA). A human-AI collaborative approach yielded superior results in EFL reading assessments compared to purely AI or expert-only methods.
Area of Science:
- Educational Measurement and Assessment
- Artificial Intelligence in Education
- Cognitive Diagnostic Assessment (CDA)
Background:
- Q-matrix construction is crucial for CDA but traditionally relies on subjective, labor-intensive methods.
- Existing methods like expert judgment and verbal report analysis present significant challenges in efficiency and consistency.
- Optimizing Q-matrix construction is essential for improving the psychometric quality of diagnostic assessments.
Purpose of the Study:
- To explore the potential of generative artificial intelligence (GenAI) in optimizing Q-matrix construction for English as a Foreign Language (EFL) reading assessments.
- To compare the psychometric performance of purely GenAI-generated Q-matrices, a human-AI collaborative Q-matrix, an expert-constructed Q-matrix, and a student-derived Q-matrix.
- To evaluate the effectiveness of different Q-matrix construction methods using various cognitive diagnostic models.
Main Methods:
- Three GenAI models (DeepSeek-V3.2, Kimi 2.5, Doubao 2.0) were used to generate Q-matrices.
- A human-AI collaborative Q-matrix was created through expert revision of a GenAI-generated matrix.
- Psychometric performance was evaluated using simulated and empirical data (N=1083 EFL learners) with G-DINA, ACDM, and RRUM models.
Main Results:
- The human-AI collaborative Q-matrix demonstrated superior performance across all models, showing the best model-data fit, classification accuracy, item parameter stability, and attribute correlation.
- Purely GenAI-informed Q-matrices showed mixed results, with some improvements in relative fit and slip stability but variable absolute fit and attribute correlation.
- The collaborative approach significantly outperformed purely AI-generated and expert-constructed matrices in key psychometric indicators.
Conclusions:
- GenAI offers a feasible and effective pathway to enhance the efficiency, consistency, and psychometric quality of Q-matrix construction in CDA.
- Human-AI collaboration in Q-matrix development holds significant promise for advancing language assessment methodologies.
- This study provides a foundational framework for leveraging AI to address methodological bottlenecks in cognitive diagnostic assessment.
