Evaluating Model Predictive Performance in Confirmatory Factor Analysis with Binary Outcomes Using the InterModel...

Area of Science:

• Psychometrics
• Statistical Modeling

Background:

• Confirmatory Factor Analysis (CFA) is standard for assessing measurement model fit.
• Existing fit indices may not fully capture a model's predictive accuracy or generalizability.

Purpose of the Study:

• Introduce the InterModel Vigorish (IMV), a novel predictive fit index for CFA.
• Evaluate the IMV's effectiveness in assessing model misspecification and generalizability, especially with binary outcomes.

Main Methods:

• Developed the InterModel Vigorish (IMV) as a predictive fit index.
• Extended IMV for use within the Confirmatory Factor Analysis (CFA) framework for binary outcomes.
• Conducted four simulation studies to assess IMV's performance and generalizability.

Main Results:

• The IMV effectively detects model misspecification at both scale and item levels.
• IMV is insensitive to sample size variations, a key advantage over traditional indices.
• The index discourages overfitting by prioritizing predictive accuracy and offers detailed item-level diagnostic information.

Conclusions:

• The InterModel Vigorish (IMV) provides valuable, interpretable insights for model comparison and evaluation in CFA.
• IMV enhances model assessment by focusing on predictive accuracy and generalizability to hold-out data.
• An accompanying R package is available to support the practical application of IMV in research.