Modeling Faking in the Multidimensional Forced-Choice Format: The Faking Mixture Model

Area of Science:

• Psychological Measurement
• Psychometrics
• Item Response Theory

Background:

• The multidimensional forced-choice (MFC) format is theorized to mitigate response distortion (faking) by presenting respondents with item blocks of matched desirability.
• However, the assumption that item-level desirability matching effectively reduces faking at the block level requires empirical investigation.

Purpose of the Study:

• To propose and validate a novel mixture item response theory model, the Faking Mixture model, designed to quantify the fakability of MFC item blocks.
• To assess the efficacy of item desirability matching in reducing faking within MFC questionnaires.

Main Methods:

• Development of a Faking Mixture model, a novel mixture item response theory approach for analyzing faking in MFC formats.
• Utilizing within-subject data from both high- and low-stakes testing contexts for model estimation.
• Conducting simulation studies to evaluate parameter recovery under diverse conditions.
• Empirical validation of the model with real-world data.

Main Results:

• Simulation results demonstrated good parameter recovery for the Faking Mixture model across various conditions.
• Empirical validation indicated that matching item desirability within blocks is a necessary but insufficient strategy for reducing faking in MFC questionnaires.
• The Faking Mixture model provides a viable method for estimating the fakability of MFC blocks.

Conclusions:

• The Faking Mixture model offers a robust statistical framework for assessing and understanding faking in multidimensional forced-choice assessments.
• Current item matching strategies in MFC formats may not adequately control for response distortion, highlighting the need for advanced analytical approaches.
• The Faking Mixture model can inform test construction processes to develop more resilient MFC questionnaires that minimize fakability.