Hierarchical models for tumor xenograft experiments in drug development

In cancer drug development, demonstrated anticancer activity in animal models is an important step to bring a promising compound to clinic. Proper design and analysis of experiments using laboratory animals have received increasing attention recently. These experiments involve informatively censored longitudinal data with small samples. The problem is further complicated because of order constraints due to the intrinsic growth of control tumors without treatment. This article proposes a Bayesian hierarchical model to analyze informatively censored longitudinal data while accounting for the parameter constraints and providing valid small sample inference. We adopt a noniterative sampling approach, the inverse Bayes formulae (IBF) sampler, to generate independent posterior samples, which avoids convergence problems associated with Markov chain Monte-Carlo methods. To effectively deal with the restricted parameter problem, we use a linear transformation to simplify the constraints and exploit the IBF method to generate random samples from truncated multivariate normal distributions. Because diffuse priors are used, the posterior modes approximate the maximum likelihood estimates well, and the hierarchical model can be considered as an extended mixed-effects model. A real xenograft experiment on a new treatment is analyzed by using the proposed method.