The problem of effect size heterogeneity in meta-analytic structural equation modeling

Area of Science:

• Psychology
• Quantitative Psychology
• Meta-analysis

Background:

• Meta-analytic structural equation modeling (MASEM) is increasingly used for theory building.
• A significant challenge is incorporating effect size heterogeneity from bivariate meta-analyses into MASEM.
• Existing MASEM methods often fail to account for this heterogeneity, limiting the generalizability of findings.

Purpose of the Study:

• To quantify the impact of ignoring effect size heterogeneity in MASEM.
• To introduce novel techniques for retaining and modeling variability in MASEM.
• To improve the representativeness of MASEM parameters and fit indices.

Main Methods:

• Quantified the problem of unaddressed heterogeneity in MASEM.
• Developed and applied techniques to retain true score relationships and their variability.
• Utilized simulated data and reanalyzed published MASEM studies.

Main Results:

• MASEM effect sizes, path estimates, and overall fit values may not generalize broadly.
• Path estimates and model fit indices are less representative of the population than previously assumed.
• The proposed techniques offer a more accurate representation of population parameters.

Conclusions:

• Ignoring effect size heterogeneity in MASEM leads to less generalizable results.
• Two extension MASEM techniques are proposed to quantify estimate stability across populations.
• These methods, usable in R or online software, enhance theory building and testing in psychology.