Detecting Misspecified Multilevel Structural Equation Models with Common Fit Indices: A Monte Carlo Study

Area of Science:

• Multilevel Structural Equation Modeling (SEM)
• Psychometrics
• Statistical Modeling

Background:

• Assessing model fit is crucial in multilevel SEM.
• Common fit indices (RMSEA, CFI, TLI, SRMR-W, SRMR-B) have varying sensitivities to misspecification.
• Understanding these sensitivities is key for accurate model evaluation.

Purpose of the Study:

• To investigate the sensitivity of common fit indices to various misspecifications in multilevel SEMs.
• To compare the performance of RMSEA, CFI, TLI, SRMR-W, and SRMR-B under different simulation conditions.
• To identify which fit indices are best suited for detecting within-group versus between-group misspecification.

Main Methods:

• A Monte Carlo simulation study was employed.
• Design factors included number of groups, group size, intra-class correlation (ICC), and misspecification types (simple, complex).
• Sensitivity of RMSEA, CFI, TLI, SRMR-W, and SRMR-B to misspecified multilevel SEMs was analyzed.

Main Results:

• CFI, TLI, and RMSEA primarily detected within-group misspecifications, particularly in pattern coefficients.
• SRMR-W was more sensitive to within-group factor covariance misspecification.
• SRMR-B was the only index sensitive to between-group misspecification, especially in factor covariance. TLI showed superior hit rates for within-group covariance misspecification.

Conclusions:

• Different fit indices have distinct sensitivities to within-group and between-group misspecifications in multilevel SEMs.
• SRMR-B is essential for detecting between-group model misspecification.
• The choice of fit index should align with the expected type of misspecification for robust multilevel SEM analysis.