An evaluation of the Rai and Van Ryzin dose-response model in teratology

The underlying assumptions of the Rai and Van Ryzin dose-response model for reproductive toxicological data are evaluated on the basis of existing experimental data. The model under consideration is unusual in its use of litter size to completely account for extra-binomial variation in the data by associating litter size with reproductive outcome. The experimental data show that controlling litter size is not sufficient to account for the litter-to-litter variability in responses. It is also shown that the two linear components of the Rai and Van Ryzin model are inappropriate. For the component which applies to the dam, the data suggest a strong nonlinearity, supported by rejection of the linear model via statistical hypothesis tests. In the component involving litter size, a relationship with dose is not apparent. The litter size parameters offer considerable potential for bias in estimation; bias which is at least partly masked by the model having good prediction characteristics due to the increased number of parameters. A simulation study is presented to illustrate how the Rai and Van Ryzin model can exaggerate litter size effects on the probability of response when the simulated data arise from a model involving a nonlinear dam component, common to this type of data, and no effect of litter size.