Cluster sampling techniques in quantal response teratology and developmental toxicity studies

This paper presents a model-free approach for evaluating teratology and developmental toxicity data involving clustered binary responses. In teratology studies, a major statistical problem arises from the effect of intralitter correlation, or the potential for littermates to respond similarly. Some statistical methods impose strict distributional assumptions to account for extra-binomial variation, while others rely on nonparametric resampling and empirical variance estimation techniques. Quasi-likelihood methods and generalized estimating equations (GEE), which model the marginal mean/variance relationship, also avoid strict distributional assumptions. The proposed approach, often used to analyze complex sample survey data, is based on a first-order Taylor series approximation and a between-cluster variance estimation procedure, yielding consistent variance estimates for binomial-based proportions and regression coefficients from dose-response models. The cluster sample technique, presented here in the context of a logistic dose-response model, incorporates many of the advantages of quasi-likelihood methods, are valid for any underlying nested correlation structure, and are adaptable to a variety of analytical settings. The results of a simulation study show the cluster sample technique to be a viable competitor to GEE methods currently receiving attention.