Computing Multivariate Effect Sizes and Their Sampling Covariance Matrices With Structural Equation Modeling: Theory,...

Area of Science:

• Social and Behavioral Sciences
• Statistical Modeling

Background:

• Reporting effect sizes and their sampling variances is crucial in social and behavioral sciences.
• While formulas for common univariate effect sizes are established, multivariate effect sizes remain less explored.
• Existing methods often assume homogeneity of variances, which may not hold in complex designs.

Purpose of the Study:

• To demonstrate the application of structural equation modeling (SEM) for computing multivariate effect sizes and their sampling covariance matrices.
• To specifically address the standardized mean difference in studies with multiple treatments and endpoints.
• To evaluate the performance of SEM under varying assumptions of variance homogeneity.

Main Methods:

• Utilizing structural equation modeling (SEM) to derive multivariate effect sizes.
• Focusing on the standardized mean difference for multiple-treatment and multiple-endpoint scenarios.
• Employing R for empirical examples and conducting two computer simulation studies to assess SEM's performance.

Main Results:

• The SEM approach provides a viable method for estimating multivariate effect sizes and their covariances.
• Simulation studies confirmed the empirical performance of the SEM approach.
• Results indicate that avoiding the homogeneity of variance assumption is preferable when it is questionable in complex studies.

Conclusions:

• Structural equation modeling offers a flexible framework for calculating multivariate effect sizes, particularly in complex research designs.
• The findings support the use of SEM for robust effect size estimation when homogeneity assumptions are violated.
• This method enhances the reporting standards for complex studies in the social and behavioral sciences.