Minimal progress toward sustainment: 10-year replication of substance use EBP sustainment trajectories and associations with implementation characteristics

Over the past decade, implementation researchers have empirically identified factors influencing long-term sustainment of evidence-based practices (EBPs) to target in implementation efforts. We examined progress toward promoting sustainment by conducting a conceptual replication of a prior study (Hunter et al., 2015, Implementation Science) that measured sustainment of an exemplar EBP for youth substance use, the Adolescent Community Reinforcement Approach (A-CRA).

Data were collected 1-5 years after initial implementation funding ended (M = 3.3 years) through interviews and surveys with clinicians and supervisors from service organizations that implemented A-CRA (n = 66). Using survival analysis, we calculated the probability of A-CRA sustainment (dichotomously reported [yes/no] in interviews) over time and examined associations with contextual factors across the multilevel domains of the Consolidated Framework for Implementation Research (CFIR). We also combined our data with Hunter et al. (n = 68) to test if sustainment status or interactions with contextual factors differed by sample, and used rapid qualitative analysis of interviews to further explore patterns in the quantitative findings.

In our sample, A-CRA sustainment probability decreased over time; 71% of organizations were sustaining A-CRA when funding ended, whereas only 33% were sustaining 5 years later; this survival curve did not statistically differ from Hunter et al. Sustainment was significantly associated with factors across CFIR domains: we replicated associations found by Hunter et al. (with e.g., funding stability, available clinicians, intervention complexity) and found unique associations (with e.g., program evaluation and strategic planning capacities, available supervisors, and perceived advantages and success of A-CRA). One association from the prior sample did not fully replicate (p < .10), but there were no significant interactions between contextual factors and sample. Qualitative findings further contextualized these results with service organization perspectives on factors influencing sustainment.

Our findings suggest that work over the past decade promoting sustainment of EBPs for youth substance use may not have produced measurable impacts. Future work needs to better incorporate growing knowledge on sustainment predictors into development and testing of robust, multilevel implementation strategies and system-level supports. This study also provides a useful illustration of a replication study in implementation science, which are important but rare.