Cohen's d needs to be readily interpretable: comment on Shieh (2013)

Area of Science:

• Statistics
• Psychometrics
• Quantitative Psychology

Background:

• Standardized effect sizes are crucial for interpreting research findings.
• Shieh (2013) proposed delta-star (δ*) as a standardized effect size for two-independent-groups designs with heteroscedasticity.
• Shieh's work focused on the inferential challenges, particularly confidence interval calculation for δ*.

Purpose of the Study:

• To evaluate the appropriateness of the standardizer used for δ*.
• To assess the consistency of δ* with conventional Cohen's d.
• To examine the generalizability and meta-analytic utility of δ*.

Main Methods:

• Conceptual analysis of effect size measures.
• Comparison of δ* with Cohen's d under heteroscedasticity.
• Evaluation of standardizer choice and its impact on interpretation and meta-analysis.

Main Results:

• The standardizer for δ* is suitable for inference but creates inconsistency with Cohen's d.
• δ* is dependent on relative sample sizes, limiting its generality.
• The proposed δ* measure is less interpretable and less suitable for meta-analysis compared to alternatives.

Conclusions:

• Researchers should prefer Cohen's d standardized by the best estimate of the population standard deviation (e.g., control population SD) over δ* when dealing with heteroscedasticity.
• The choice of standardizer significantly impacts the interpretability and meta-analytic application of effect size measures.
• Alternative standardization strategies for Cohen's d are recommended for heteroscedastic conditions.