Accommodating negative intracluster correlation with a mixed effects logistic model for bivariate binary data

We extend the random intercept logistic model to accommodate negative intracluster correlations for bivariate binary response data. This approach assumes a single random effect per cluster, but entails separate affine transformations of this random effect for the two responses of the pair. We show this approach works for two data sets and a simulation, whereas other mixed effects approaches fail. The two data sets are from a crossover trial and a developmental toxicity study of the effects of chemical exposure on malformation risk among rat pups. Comparisons are made with the conditional likelihood approach and with generalized estimating equations estimation of the population-averaged logit model. Simulations show the conditional likelihood approach does not perform well for moderate to strong negative correlations, as a positive intracluster correlation is assumed. The proposed mixed effects approach appears to be slightly more conservative than the population-averaged approach with respect to coverage of confidence intervals. Nonetheless, the statistical literature suggests that mixed effects models provide information in addition to that provided by population-averaged models under scientific contexts such as crossover trials. Extensions to trivariate and higher-dimensional responses also are addressed. However, such extensions require certain constraints on the correlation structure.