A comparison of recent methods for the analysis of small-sample cross-over studies

The standard analysis of variance (ANOVA) method is usually applied to analyse continuous data from cross-over studies. The method, however, has been known to be not robust for general variance-covariance structure. The simple empirical generalized least squares (EGLS) method, proposed in an attempt to improve the precision of the standard ANOVA method for general variance-covariance structure, is usually insufficient for small-sample cross-over trials. In this paper we compare the following commonly used or recent approaches: standard ANOVA; simple EGLS; modified ANOVA method derived from a modified approximate F-distribution; and a modified EGLS method adjusted by the Kenward and Roger procedure in terms of robustness and power while applying to small-sample cross-over studies (say, the sample size is less than 40) over a variety of variance-covariance structures by simulation. We find that the unconditional modified ANOVA method has robust performance for all of the simulated small-sample cross-over studies over the various variance-covariance structures, and has comparable power with the standard ANOVA method whenever they are comparable in type I error rate. The EGLS method (simple or modified) is not reliable when the sample size of a cross-over study is too small, say, less than 24 in the simulation, unless a simple covariance structure is correctly assumed. Given a relatively larger sample size, the modified EGLS method, assuming an unstructured covariance matrix, demonstrates robust performance over the various variance-covariance structures in the simulation and provides more powerful tests than those of the modified (or standard) ANOVA method.