Multidimensional IRT for forced choice tests: A literature review

Area of Science:

• Psychometrics
• Educational Measurement
• Psychological Assessment

Background:

• Multidimensional Forced Choice (MFC) tests are widely used in non-cognitive assessments to mitigate response biases inherent in Likert scales.
• However, MFC tests produce ipsative data, which presents challenges for individual comparisons and normative data development.
• Recent advancements in Multidimensional Item Response Theory (MIRT) offer promising solutions for analyzing forced-choice data.

Purpose of the Study:

• To introduce a novel modeling framework for Multidimensional Forced Choice Item Response Theory (MFC-IRT) integrating response format, measurement model, and decision theory.
• To compare parameter estimation techniques within MFC-IRT models.
• To examine empirical applications of MFC-IRT in parameter invariance testing, computerized adaptive testing (CAT), and validity investigations.

Main Methods:

• Development of a conceptual framework for MFC-IRT analysis.
• Selection and application of four example IRT models within the framework.
• Comparative analysis of parameter estimation techniques for MFC-IRT models.
• Empirical examination of MFC-IRT in parameter invariance, CAT, and validity studies.

Main Results:

• The study provides a structured framework for understanding and applying MFC-IRT models.
• It offers a comprehensive comparison of parameter estimation methods relevant to MFC data.
• Empirical analyses demonstrate the potential of MFC-IRT in advancing psychometric applications.

Conclusions:

• The proposed MFC-IRT framework enhances the utility of forced-choice tests by enabling normative data collection and robust psychometric analysis.
• Future research should focus on refining MFC-IRT models, parameter invariance testing, forced-choice CAT, and validity studies.
• These advancements are crucial for improving the accuracy and applicability of non-cognitive assessments.