Does Cluster-Robust Estimation Provide Within-Study Effects? A Comparison of Individual Participant Data Methods in...

Area of Science:

• Psychometrics
• Statistical Modeling
• Meta-Analysis

Background:

• Individual participant data meta-analysis (IPD MASEM) offers advanced modeling capabilities.
• Several methods exist for IPD MASEM, including cluster-robust estimation, two-level SEM, and One-Stage MASEM (OSMASEM).
• Cluster-robust estimation is popular but may produce divergent results compared to other techniques.

Purpose of the Study:

• To compare the performance of different IPD MASEM methods.
• To evaluate the accuracy and biases associated with cluster-robust estimation versus other approaches.
• To provide guidance on selecting appropriate IPD MASEM methods.

Main Methods:

• The study employed simulated data for meta-analytical structural equation modeling (MASEM).
• Simulations varied key factors: intraclass correlations, parameter equality, number of studies, and missing data.
• Performance was assessed by comparing within-study estimates, standard errors, and model fit across methods.

Main Results:

• Cluster-robust estimation frequently misrepresented within-study estimates.
• Biased standard errors were commonly observed with cluster-robust estimation.
• Cluster-robust estimation tended to incorrectly reject model fit more often than other methods.

Conclusions:

• Cluster-robust estimation may not be suitable for all IPD MASEM applications due to potential biases.
• The findings underscore the importance of method selection in IPD MASEM.
• Researchers should carefully consider alternative methods to ensure accurate meta-analytical structural equation modeling.