Do we consider scalability from the outset? A methodological review of pilot randomised controlled health trials

Health interventions should be designed to be appropriate for scaling from the outset, but the extent to which factors that are important for scalability are reported on in pilot randomised trials is unclear. This review assesses the extent to which pilot randomised trials report on 15 domains of intervention scalability.

Methodological review.

Four journals were searched: BMJ Open, BMC Pilot and Feasibility Studies, BMC Trials and PLoS One for articles published between January 2023 and October 2024.

We included pilot randomised trials of health interventions.

Data relevant to 15 scalability domains, derived from the Intervention Scalability Assessment Tool and wider implementation science literature, were extracted. Data were double-extracted for 20% of the included studies. Two authors scored all studies from 0 to 3 (0=Not at all; 1=Small extent; 2=Moderate extent; 3=Large extent) on the extent to which each of the 15 scalability domains was reported. For each scalability domain, we calculated the mean score and the frequency of each categorical score across the included studies.

Titles and abstracts screening (521 publications) resulted in 132 full-text publications for review. Of the 104 eligible studies, a random sample of 50 studies were selected for detailed review. Through snowballing, an additional 49 associated publications were identified (eg, protocols), resulting in 99 publications across 50 studies. Most studies reported the Problem (30/50, 60%; mean=2.4 ± 0.8) and the Intervention (37/50, 74%; mean=2.7 ± 0.4) to a large extent (ie, scored 3), with the Delivery Setting and Workforce domain most often receiving a score of 2 (28/50, 56%; mean=2.2 ± 0.7). For eight of the scalability domains, the majority of studies scored 0.

The extent of scalability domain reporting in pilot trials of health interventions is limited. Explicit consideration of scalability in pilot trials could improve the design of fully powered randomised controlled trials and enhance the potential for effective interventions to be translated into practice. Future research should consider if and how to incorporate scalability considerations into pilot trials, and whether pilot trial and intervention reporting guidelines should be expanded to include scalability considerations.