Equivalence testing to judge model fit: A Monte Carlo simulation.

Area of Science:

• Psychometrics
• Statistical Modeling
• Quantitative Psychology

Background:

• Traditional structural equation model (SEM) fit indices like chi-square, CFI, and RMSEA demonstrate inconsistent and unreliable performance.
• Researchers lack robust inferential alternatives for assessing SEM fit, often relying on these problematic indices.
• Equivalence testing adaptations of RMSEA and CFI (RMSEA_eq, CFI_eq) were proposed as a potential inferential solution.

Purpose of the Study:

• To empirically evaluate the accuracy of equivalence testing for SEM fit indices (RMSEA_eq, CFI_eq).
• To assess the reliability of equivalence testing across various conditions, including sample size, model specification, and data characteristics.
• To determine if equivalence testing provides a more accurate judgment of acceptable and unacceptable model fit compared to traditional methods.

Main Methods:

• A fully crossed Monte Carlo simulation was employed.
• Evaluated equivalence testing accuracy under diverse conditions: sample size (100-1,000), model specification (correct/misspecified), model type (CFA, path analysis, SEM), variable load, data distribution (normal/skewed), and missing data (0-25%).
• Used proportional z-tests and logistic regression to analyze results.

Main Results:

• Equivalence testing demonstrated inconsistent and unreliable performance across various independent variable conditions.
• The accuracy of RMSEA_eq and CFI_eq was often contingent on complex interactions between simulation conditions.
• Equivalence tests for SEM fit were found to be problematic, particularly under conditions of mild model misspecification.

Conclusions:

• Current equivalence testing approaches for SEM fit indices (RMSEA_eq, CFI_eq) are not consistently reliable.
• The effectiveness of these tests is highly dependent on specific model and data characteristics, especially misspecification.
• Researchers should exercise caution when using equivalence testing for SEM fit until further research and development address its limitations.