Redefining effect size interpretations for psychotherapy RCTs in depression

Effect sizes are often used to interpret the magnitude of a result and in power calculations when planning research studies. However, as effect size interpretations are context-dependent, Jacob Cohen's suggested guidelines for what represents a small, medium, and large effect are unlikely to be suitable for a diverse range of research populations and interventions. Our objective here is to determine empirically-derived effect size thresholds associated with psychotherapy randomized controlled trials (RCTs) in depression by calculating the effect size distribution.

We extracted effect sizes from 366 RCTs provided by the systematic review of Cuijpers and colleagues (2020) on psychotherapy for depressive disorders across all age groups. The 50th percentile effect size, as this represents a medium effect size, and the 25th (small) and 75th (large) percentile effect sizes were calculated to determine empirically-derived effect size thresholds.

After adjusting for publication bias, 0.27, 0.53, and 0.86 represent small, medium, and large effect sizes, respectively, for psychotherapy treatment for depressive disorders.

The effect size distribution for psychotherapy treatment of depression indicates that observed effect size thresholds are larger than Cohen's suggested effect size thresholds (0.2, 0.5, and 0.8). These results have implications for the interpretation of study effects and the planning of future studies via power analyses, which often use effect size thresholds.