Diagnostic Classification Models for Testlets: Methods and Theory

Area of Science:

• Educational Measurement
• Psychometric Modeling
• Latent Variable Analysis

Background:

• Diagnostic Classification Models (DCMs) are crucial for formative assessment.
• Testlet Response Theory (TRT) models, like the testlet DINA (T-DINA), incorporate item grouping effects.
• Existing T-DINA models assume independence between attribute profiles and testlet effects.

Purpose of the Study:

• To extend the T-DINA model by allowing for correlations between attribute profiles and testlet effects.
• To investigate the identifiability of the proposed extended T-DINA model.
• To evaluate the model's performance using real-world assessment data.

Main Methods:

• Development of an extended testlet DINA (T-DINA) model incorporating correlated latent structures.
• Theoretical analysis of model identifiability, establishing sufficient conditions.
• Application of the model to the 2015 Programme for International Student Assessment (PISA) dataset.
• Comparative analysis with standard DINA and T-DINA models.
• Simulation studies to assess model performance under various conditions.

Main Results:

• The proposed extended T-DINA model demonstrates substantial improvements in goodness-of-fit compared to DINA and standard T-DINA.
• Sufficient conditions for the identifiability of the extended model were established.
• The identifiability of the standard T-DINA model was also confirmed as a secondary outcome.
• The model showed robust performance in simulation studies across different settings.

Conclusions:

• The extended T-DINA model offers a more accurate representation of complex data structures in educational and psychological measurement.
• Accounting for the correlation between attribute profiles and testlet effects enhances model fit and provides deeper insights.
• The findings support the use of this advanced DCM for improved formative assessment and data analysis.