Estimating heterogeneity of treatment effect in psychiatric clinical trials

Currently, placebo-controlled clinical trials report mean change and effect sizes, which masks information about heterogeneity of treatment effects (HTE). Here, we present a novel method to estimate HTE and evaluate the null hypothesis (H₀) that a drug has equal benefit for all participants (HTE = 0). We developed a measure termed 'estimated heterogeneity of treatment effect' or eHTE, which compares the distributions between study arms to test for heterogeneity in drug response - rather than testing if any specific covariate impacts response. First, we used simulations to test fidelity, sensitivity and power of eHTE to detect heterogeneity. We found that eHTE accurately estimated heterogeneity in many cases (though under-estimated heterogeneity at times) and could detect realistic heterogeneity in trials around N = 200. Then, the approach was tested across numerous large placebo-controlled clinical trials. In contrast with variance-based methods which have not identified heterogeneity in psychiatric trials, reproducible instances of treatment heterogeneity were found. For example, heterogeneous response was found in a trial of venlafaxine for depression (p_eHTE = 0.034), and two trials of dasotraline for binge eating disorder (Phase 2, p_eHTE = 0.002; Phase 3, 4 mg p_eHTE = 0.011; Phase 3, 6 mg p_eHTE = 0.003). Significant response heterogeneity was detected in other datasets as well, often despite no difference in variance between placebo and drug arms. The implications of eHTE as a clinical trial outcomes, independent of central tendency of the group, is considered and the importance of the eHTE method and results for drug developers, providers, and patients is discussed.