Testing for qualitative interactions between stages in an adaptive study

I consider the underlying structure for a test of qualitative interaction of a treatment when assessing heterogeneity between stages in an adaptive trial. Since decisions about the clinical utility of a drug are based on the balance of risks and benefits, a quantitative interaction in treatment efficacy across different groups could lead to qualitatively different decisions. Thus, the difference between quantitative and qualitative interactions is not a true dichotomy. I show that the standard tests for qualitative interactions (Gail and Simon,Biometrics 1985; 41:361-372; Piantadosi and Gail, Statist. Med. 1993; 12:1239-1248) are very conservative in this application. Theoretical calculations in a simpler situation confirm that the published criteria are very conservative, which may help explain why the tests are known to have very low power to detect interaction. I introduce the concept of 'minimum detectable effect', which is the smallest effect that a study could identify as statistically significant. I propose that important heterogeneity between stages in an adaptive trial be identified when two criteria are met. First, at least one individual stage must be below the overall study mean by at least the minimum detectable effect. Second, using an appropriate critical value based on simulations, there must be statistically significant heterogeneity between the stages.