Effect size measures in a two-independent-samples case with nonnormal and nonhomogeneous data

Area of Science:

• Psychological Science
• Statistical Methods
• Quantitative Psychology

Background:

• The
• new statistics
• advocate for effect size (ES) estimation over null-hypothesis significance testing.
• Cohen's d is a popular ES but sensitive to normality and homogeneity of variance assumptions.
• Alternative ES measures exist but lack systematic performance evaluation under assumption violations.

Purpose of the Study:

• To systematically evaluate the performance of six effect size measures under violations of normality and homogeneity of variances.
• To identify robust effect size estimators for psychological research.

Main Methods:

• A simulation study was conducted examining five factors: data distribution, sample type, base rate, variance ratio, and sample size.
• Performance of Cohen's d, unscaled robust d (d_r^*), scaled robust d (d_r), point-biserial correlation (r_pb), common-language ES (CL), and nonparametric CL estimator (A_w) was assessed.

Main Results:

• The scaled robust d (d_r) and the nonparametric common-language ES estimator (A_w) demonstrated general robustness to violations of normality and homogeneity of variances.
• A_w slightly outperformed d_r in terms of robustness across the simulated conditions.

Conclusions:

• A_w and d_r are recommended as robust alternatives to Cohen's d when assumptions are questionable in psychological research.
• The findings support the broader adoption of robust effect size measures in empirical studies.