Multilevel bootstrap analysis with assumptions violated

Likelihood-based methods can work poorly when the residuals are not normally distributed and the variances across clusters are heterogeneous.

The performance of two estimation methods, the non-parametric residual bootstrap (RB) and the restricted maximum likelihood (REML) for fitting multilevel models are compared through simulation studies in terms of bias, coverage, and precision.

We find that (a) both methods produce unbiased estimates of the fixed parameters, but biased estimates of the random parameters, although the REML was more prone to give biased estimates for the variance components; (b) the RB method yields substantial reductions in the difference between nominal and actual confidence interval coverage, compared with the REML method; and (c) for the square root of the mean squared error (RMSE) of the fixed effects, the RB method performed slightly better than the REML method. For the variance components, however, the RB method did not offer a systematic improvement over the REML method in terms of RMSE.

It can be stated that the RB method is, in general, superior to the REML method with violated assumptions.