Comparing multiple people each to the grand mean of log-normally distributed endpoints

Multiple anesthesiology endpoints follow log-normal distributions, including surgical times, anesthesia times, epidural placement times, extubation times, and post-anesthesia care unit times. These can be compared among people (e.g., anesthesiologists or surgeons), hospitals, or clinical trial centers to detect outliers. We consider comparisons of each group, henceforth referred to as a "person," with the grand mean.

Monte-Carlo simulations were performed to evaluate risks of false detection (Type I errors) using generalized linear models, generalized pivotal methods, and mixed effects models with empirical Bayes means and standard errors predicted from the random effects. Models were evaluated with and without heteroscedastic consistent (robust) standard errors.

Errors were > 10-fold when fixed effects models lacked Bonferroni correction for multiple comparisons. Šidák adjustments had small incremental inaccuracy because the comparisons being made were not statistically independent of one another. Robust variance estimates were more accurate than the default observed information matrix method. However, the sample size of 10 cases per person was too small for the robust variance method, resulting in large inaccuracy. With Bonferroni adjustment, robust variance estimation, and log link, there also were large errors with unequal sample sizes, even with a median of 60 cases per person. When a mixed effects model was used instead (i.e., shrinkage estimates), Bonferroni adjustment yielded highly inaccurate results. Without adjustment, errors were small to moderate. Generalized pivotal inference applied individually for each person compared with the grand mean had very small errors when used with Šidák adjustment.

For log-normally distributed continuous outcomes, with sample sizes of 100's of cases per person, consider fixed effect models with Bonferroni adjustment and robust variance estimation. Otherwise, we recommend generalized pivotal inference with Šidák adjustment.