Model Selection for Cogitative Diagnostic Analysis of the Reading Comprehension Test

Area of Science:

• Educational Measurement
• Psychometrics

Background:

• Traditional reading diagnosis models often assume dichotomous latent variables, which may not accurately reflect continuous reading subskills.
• Multidimensional item response theory (MIRT) models, with their continuous latent variables, offer a potential alternative for more nuanced reading diagnostics.

Purpose of the Study:

• To compare the performance of the MIRT model against two traditional models, the reduced reparametrized unified model (R-RUM) and the generalized deterministic, noisy, and gate (G-DINA) model, for reading diagnoses.
• To evaluate which model provides more accurate diagnostic information regarding reading subskill mastery.

Main Methods:

• Model-data fit indices were used to compare MIRT, R-RUM, and G-DINA with empirical data.
• Simulated data were used to compare the accuracy of estimated scores, correct classification rates, and frequency distributions of subskill mastery probabilities among the three models.

Main Results:

• The MIRT model demonstrated superior model-data fit compared to R-RUM and G-DINA.
• MIRT yielded estimated scores that better represented true abilities and showed higher correct classification rates.
• MIRT exhibited less deviation in subskill mastery probability distributions, indicating more precise diagnostic information.

Conclusions:

• The MIRT model provides more accurate and reasonable diagnostic information for reading abilities than R-RUM and G-DINA.
• The enhanced accuracy of MIRT has significant implications for guiding remedial teaching and improving learning outcomes.
• MIRT is recommended as a superior methodology for future reading diagnostic analyses.