Understanding site variability in a multisite clinical trial of a technology-delivered psychosocial intervention for...

Significant fixed effects of site (main effects of site and/or site by treatment interactions) on primary outcome have been identified in the majority of studies performed by NIDA's National Drug Abuse Treatment Clinical Trials Network. While rarely explored, identifying patient- and site-level variables that are associated with site effects can provide information about the context in which outcome is optimized.

In a 10-site clinical trial that evaluated the effectiveness of a web-based psychosocial intervention compared to usual treatment of patients (N = 507) with substance use disorders, the primary outcome analysis revealed significant main effect of site, modeled as a fixed effect, on the outcome of abstinence (Campbell et al., 2014). In the current analysis, we use a two-level, hierarchical generalized linear model (HGLM) to identify patient- and site-level variables associated with abstinence outcome, while modeling site as a random factor.

The site-specific percentage of patients abstinent in the last 4 weeks of the study varied from 6.1% to 40%. However, only 6.7% (p = 0.08) of variability in end-of-study abstinence was accounted for by site, indicating a small-moderate effect. Among patient-level predictors, older age (OR = 1.40; 95% CI = 1.15, 1.71; p = 0.0009), abstinence at baseline (OR = 2.77; 95% CI = 1.73, 4.45; p < 0.0001), and among site-level predictors, higher annual clinic admissions (OR = 1.28; 95% CI = 1.03, 1.59; p = 0.0251) were significantly associated with increased likelihood of abstinence. When controlling for these three variables in a HGLM, only patient age and abstinence at baseline remained significant, and random factor site explained only 1.4% of variability in end-of-study abstinence, a 79% reduction in magnitude.

The findings suggest that only some amount of variability in abstinence outcomes among sites can be explained by a combination of patient- and site-level variables. Our findings support the case that variability between sites is a natural phenomenon, and our methodological recommendation is that site be modeled as a random factor when analyzing multi-site clinical trials.