Testing the Performance of Level-Specific Fit Evaluation in MCFA Models With Different Factor Structures Across...

Area of Science:

• Multilevel Confirmatory Factor Analysis (MCFA)
• Statistical Modeling
• Psychometrics

Background:

• Assessing model fit is crucial in MCFA.
• Previous research primarily focused on simpler MCFA structures.
• The performance of fit evaluation methods under complex, multi-level factor structures requires further investigation.

Purpose of the Study:

• To compare level-specific (LS) and simultaneous (SI) fit evaluation performance in MCFA.
• To examine performance across MCFA models with varying factor structures at different levels.
• To investigate the influence of intraclass correlation (ICC) and misspecification type (MT) on fit evaluation.

Main Methods:

• A Monte Carlo simulation study was employed.
• MCFA models with different factor structures across levels were simulated.
• The impact of ICC, group size (GS), and MT on LS and SI fit evaluation was analyzed.

Main Results:

• LS fit evaluation demonstrated superior performance over SI in detecting between-group misspecification.
• The effectiveness of LS fit indices was contingent upon ICC, GS, and MT.
• Root Mean Square Error of Approximation (RMSEA) performance improved with increased GS and ICC for between-level models.
• Standardized Root Mean Squared Residual (SRMR) performance enhanced with higher ICC, particularly for structural misspecification.

Conclusions:

• LS fit evaluation is recommended for detecting misspecification in MCFA, especially at the between-group level.
• Researchers should consider ICC, GS, and MT when interpreting LS fit indices.
• Findings provide valuable guidance for model evaluation in complex multilevel data structures.