Trial Characteristics and Appropriateness of Statistical Methods Applied for Design and Analysis of Randomized...

To evaluate whether clustering effects, often quantified by the intracluster correlation coefficient (ICC), were appropriately accounted for in design and analysis of school-based trials.

We searched PubMed and extracted variables concerning study characteristics, power analysis, ICC use for power analysis, applied statistical models, and the report of the ICC estimated from the observed data.

N=263 papers were identified, and N=121 papers were included for evaluation. Overall, only a minority (21.5%) of studies incorporated ICC values for power analysis, fewer studies (8.3%) reported the estimated ICC, and 68.6% of studies applied appropriate multilevel models. A greater proportion of studies applied the appropriate models during the past five years (2013-2017) compared to the prior years (74.1% versus 63.5%, p=0.176). Significantly associated with application of appropriate models were a larger number of schools (p=0.030), a larger sample size (p=0.002), longer follow-up (p=0.014), and randomization at a cluster level (p < 0.001) and so were studies that incorporated the ICC into power analysis (p=0.016) and reported the estimated ICC (p=0.030).

Although application of appropriate models has increased over the years, consideration of clustering effects in power analysis has been inadequate, as has report of estimated ICC. To increase rigor, future school-based trials should address these issues at both the design and analysis stages.