Ignoring the group in group-level HIV/AIDS intervention trials: a review of reported design and analytic methods

Studies evaluating the efficacy of HIV/AIDS interventions often involve the random assignment of groups of participants or the treatment of participants in groups. These studies require analytic methods that take within-group correlation into account. We reviewed published studies to determine the extent to which within-group correlation was dealt with properly.

We reviewed group-randomized trials (GRTs) and individually randomized group treatment (IRGT) trials published in HIV/AIDS and general public health journals 2005-2009.

At least two of the authors reviewed each article, recording descriptive characteristics, sample size estimation methods, analytic methods, and judgments about whether the methods took intraclass correlation into account properly.

Of those articles including sufficient information to judge whether analytic methods were correct, only 24% used only appropriate methods for dealing with the intraclass correlation. The percentages differed substantially for GRTs (41.7%) and IRGT trials (8.0%). Most of the articles (69.2%) also made no mention of a priori sample size estimation.

A majority of the articles in our review reported analyses ignoring the intraclass correlation. This practice may result in underestimated variance, inappropriately small P values, and incorrect conclusions about the effectiveness of interventions. Previous trials that were analyzed incorrectly need to be re-analyzed, and future trials should be designed and analyzed with appropriate methods. Also, journal reviewers and editors need to be aware of the special requirements for design and analysis of GRTs and IRGT trials and judge the quality of articles reporting on such trials according to appropriate standards.