Bayesian methods for the analysis of small sample multilevel data with a complex variance structure

Inferences from multilevel models can be complicated in small samples or complex data structures. When using (restricted) maximum likelihood methods to estimate multilevel models, standard errors and degrees of freedom often need to be adjusted to ensure that inferences for fixed effects are correct. These adjustments do not address problems in estimating variance/covariance components. An alternative to the adjusted likelihood method is to use Bayesian methods, which can produce accurate inferences about fixed effects and variance/covariance parameters. In this article, the authors contrast the benefits and limitations of likelihood and Bayesian methods in the estimation of multilevel models. The issues are discussed in the context of a partially clustered intervention study, a common intervention design that has been shown to require an adjusted likelihood analysis. The authors report a Monte Carlo study that compares the performance of an adjusted restricted maximum likelihood (REML) analysis to a Bayesian analysis. The results suggest that for fixed effects, the models perform equally well with respect to bias, efficiency, and coverage of interval estimates. Bayesian models with a carefully selected gamma prior for the variance components were more biased but also more efficient with respect to estimation of the variance components than the REML model. However, the results also show that the inferences about the variance components in partially clustered studies are sensitive to the prior distribution when sample sizes are small. Finally, the authors compare the results of a Bayesian and adjusted likelihood model using data from a partially clustered intervention trial.