The performance of the O'Brien-Fleming multiple testing procedure in the presence of intraclass correlation

Assuming that all subject responses were independent, O'Brien and Fleming (1979, Biometrics 35, 549-556) proposed a simple and useful multiple testing procedure for clinical trials for comparing two treatments with dichotomous data. Differences in the methods of evaluating subject responses at each evaluation time, however, may induce an intraclass correlation among these responses used in calculating the O'Brien-Fleming multiple testing procedure. On the basis of Monte Carlo simulations, we note that even a small intraclass correlation among subject responses in the same analysis can substantially inflate the Type I error of the O'Brien-Fleming multiple testing procedure. Furthermore, this inflation generally increases as either the number of analyses or the underlying response probability increases. We also have demonstrated that if we were able to maintain a uniform medical test procedure between the two treatments for each analysis, the actual Type I error of the O'Brien-Fleming multiple testing procedure may conversely become conservative.