A general diagnostic modelling framework for forced-choice assessments

Area of Science:

• Psychometrics
• Educational Psychology
• Psychological Assessment

Background:

• Diagnostic Classification Models (DCM) are used to assess strengths and weaknesses.
• DCM is increasingly applied to noncognitive traits, facing challenges like response biases.
• The forced-choice (FC) item format was adapted into DCM (FC-DCM) to mitigate biases, but has limitations.

Purpose of the Study:

• To introduce a general diagnostic framework for FC assessments within DCM.
• To adapt the G-DINA model for FC responses and evaluate its performance.
• To provide practical recommendations for using FC format in noncognitive trait assessments.

Main Methods:

• An adaptation of the G-DINA model was developed to handle FC responses.
• Simulations were conducted to compare the G-DINA model with the FC-DCM.
• A real FC assessment dataset was used to demonstrate model fit.

Main Results:

• The G-DINA model demonstrated accurate classifications, parameter estimates, and attribute correlations.
• The G-DINA model outperformed the FC-DCM, especially in scenarios with varying item discrimination.
• The G-DINA model showed a better model fit in a real FC assessment example.

Conclusions:

• The adapted G-DINA model provides a robust framework for diagnostic classification modeling of FC assessments.
• This approach enhances the assessment of noncognitive traits by addressing response biases more effectively.
• The findings support the use of the G-DINA model for improved diagnostic accuracy in FC-based noncognitive assessments.