Small numbers of clusters in cluster-randomized trials: a scoping review of problems and proposed solutions

Cluster-randomized trials (CRTs) are commonly used to evaluate health service interventions, quality improvement programmes or public health initiatives that are implemented at a regional or organizational level. Many CRTs include only a limited number of clusters for practical reasons. Small numbers of clusters are associated with known analytical and design challenges. Identifying the breadth and diversity of problems associated with small numbers of clusters and potential solutions will help researchers plan more effective evaluations.

We systematically searched the literature including methodological papers, trial papers, and commentaries where the words "small numbers of clusters" or similar expression were close to words like "problem" or "solution" for any type of CRT. We identified 53 references from the Ovid, MedLine, and Embase databases.

Most references focus either on parallel-group CRTs or on stepped-wedge CRTs (17 (32.1%) references each). Challenges related to type I error (T1E) rate inflation or other inferential issues are identified in 32 (60.4%) references, and the risk of covariate imbalance between trial arms in 30 (56.6%), making these the most commonly mentioned problems. A variety of methods to control T1Erate inflation have been proposed, but with some concerns raised over their performance down to very small numbers of clusters. Other challenges identified relate to the design or conduct (eg, generalisability of results or cluster attrition) for which no solution was proposed in the included references. Operational definitions of a "small" number of clusters vary considerably across the literature from 3 to 50 (median 20). Most published stepped-wedge trials, in particular, are what most methodological references would call small (in terms of number of clusters).

Work on mitigating the problems arising from small numbers of clusters has tended to focus on the analysis of CRTs rather than their design (eg, sample size justification) or conduct. Further work is still needed to develop simple, practical guidance to help triallists understand what challenges they will face and when, and how to optimize the performance of methods for CRTs with limited numbers of clusters.