Choice of test for comparing two groups, with particular application to skewed outcomes

We consider a clinical trial where a skewed outcome variable is to be compared between two groups. While comparison of sample means may lack power, we show that power also depends on the nature of the anticipated treatment effect. For any given distribution in the control arm, there is a family of true distributions in the intervention arm for which the most powerful test is a comparison of arithmetic means. Similar results hold for a comparison of geometric means, and approximately for the Wilcoxon rank sum test and a comparison of medians. We discuss how these methods could be used in planning the analysis of a clinical trial in which the intervention effect alters the shape of the distribution. These ideas are illustrated by a trial in community psychiatry, where the primary outcome (days in hospital) was highly skewed but the intervention was mainly expected to reduce the frequency of values in the tail. We show that a comparison of sample means is a reasonable choice in this case despite the skewness.