The performance of small sample correction methods for controlling type I error when analyzing parallel cluster...

Most cluster randomized trials (CRTs) include fewer than 50 clusters, yet the assumption of a "large sample" is relied upon to derive the sampling distributions of treatment effects. We review the current simulation study literature pertaining to small sample corrections for common analytical approaches for parallel CRTs.

We searched Ovid Medline and Web of Science up to August 30 2024 for simulation studies evaluating the performance of small sample corrections. We include binary and continuous outcomes analyzed using generalized linear mixed models, generalized estimating equations, or cluster-level approaches. Full-text screening and data abstraction was performed independently in duplicate.

Fourteen studies evaluated binary outcomes and 6 studies continuous outcomes. The number of clusters ranged from 4 to 200; the median smallest intracluster correlation coefficient was 0.001 [range 0.000-0.200]; largest intracluster correlation coefficient was 0.10 [range 0.05-0.70]; lowest prevalence was 0.25 [range 0.05-0.50]; and median coefficient of variation of cluster sizes 1.00 [range: 0.80-1.50]. For continuous outcomes, a cluster-level analysis (either unweighted or inverse-variance weighted) with a t-distribution (with between-within degree of freedom); a linear mixed model with a Satterthwaite correction; or a generalized estimating equation with the Fay and Graubard correction mostly preserve nominal type I error with as few as six clusters (although up to 40 clusters in some settings). Other approaches work less favorably (eg, Kenward-Roger is conservative even with 30 clusters). For binary outcomes, an unweighted or inverse-variance weighted cluster-level analysis can achieve nominal type I error (but can be anticonservative with small cluster sizes or low prevalence); as can a generalized linear mixed model with a between-within correction with as few as 10 clusters (but sometimes conservative with up to 30 clusters). Other corrections such as the Kenward-Roger or Satterthwaite correction are more conservative. For generalized estimating equations, the Mancl and DeRouen correction mostly seems to preserve nominal errors but can be anticonservative.

The literature on the performance of small sample corrections for parallel CRTs is complex. While the available corrections can maintain type I error with a very small number of clusters, more than 40 clusters are required to guarantee nominal type I error across all settings.