A non-parametric statistical test of null treatment effect in sub-populations

Randomized clinical trials are designed to estimate the average treatment effect (ATE). If heterogeneity of treatment effect exists, then it is possible that there may be subjects who derive a treatment effect different from the ATE. We propose a method to test the hypothesis that there exist subjects who derive benefit (or harm) against the null hypothesis that the treatment has no benefit (or harm) on each of the smallest sub-populations defined by discrete baseline covariates. Our approach is nonparametric, which generates the null distribution of the test statistic by the permutation principle. A key innovation of our method is that stochastic simulation is built into the test statistic to detect signals that may not be linearly related to the multiple covariates. This is important because, in many real clinical problems, the treatment effect is not linearly correlated with relevant baseline characteristics. We applied the method to a real randomized study that compared the Implantable Cardioverter Defibrillator (ICD) with conventional medical therapy in reducing total mortality in a low ejection fraction population. Simulations and power calculations were performed to compare the proposed test with existing methods.