Comparison among cognitive diagnostic models for the TIMSS 2007 fourth grade mathematics assessment

Area of Science:

• Educational Measurement
• Psychometrics
• Cognitive Psychology

Background:

• Cognitive diagnostic models (CDMs) are increasingly used for student assessment and evaluation.
• Different CDMs rely on distinct assumptions regarding the relationship between student skills and achievement.
• Empirical validation is crucial to determine which CDMs best represent actual student data.

Purpose of the Study:

• To comparatively evaluate the data-fitting capabilities of various representative CDMs.
• To investigate the performance of different CDMs using large-scale international assessment data.

Main Methods:

• Comparative analysis of selected Cognitive Diagnostic Models (CDMs).
• Application of CDMs to Trends in International Mathematics and Science Study (TIMSS) 2007 data.
• Analysis across seven participating countries to ensure generalizability.

Main Results:

• Cognitive diagnostic models demonstrated superior fit compared to traditional item response theory models.
• Main effects models generally outperformed other parsimonious and saturated CDMs in data fit.
• Traditional parsimonious models, including DINA and DINO, exhibited suboptimal fit.

Conclusions:

• Cognitive diagnostic models offer advantages over item response theory for educational assessment.
• Main effects models represent a more robust approach within CDMs for analyzing student skills.
• Findings highlight the need to carefully select CDMs based on empirical data fit for accurate educational evaluation.