An Implementation Log for quantitatively tracking key implementation variables in near real time: Development,...

Implementation is complex and not always reported in detail, presenting barriers to replication and translatability. The goals herein were to: (1) describe the development of an Implementation Log that yields quantitative data on a wide range of key implementation variables through near real time tracking; (2) gather qualitative feedback from study personnel, and (3) demonstrate ways the log data could be analyzed.

These goals were executed in the context of a parent hybrid effectiveness-implementation cluster randomized controlled trial to evaluate a sleep treatment (Transdiagnostic Intervention for Sleep and Circadian Dysfunction) implemented in community mental health centers via facilitation. The Implementation Log was derived from prior research on implementation tracking; reporting guidelines, frameworks, and taxonomies; piloting; and collaboration with facilitators in the trial. After development, the log was used for 17 months by study personnel (i.e., facilitators; N=6). Then, the facilitators were interviewed about their perceptions of the log's acceptability, appropriateness, and feasibility. Interviews were deductively and inductively coded. To demonstrate possible uses for the log, two sets of preliminary analyses were conducted. First, we quantified the most time-intensive events, targets, implementation strategies, and intended outcomes tracked by facilitators. Second, we quantified the most time-intensive events, implementation strategies, and targets directed toward the most time-intensive outcomes, to unpack relationships between implementation variables.

Facilitators tracked almost 4,000 hours of events, targets, and outcomes using the Implementation Log. Facilitators appreciated the log's value to the field but had three main suggestions: (1) automatize the format, (2) offer more training and supervision, and (3) improve flexibility to adjust the log's variables. Per quantitative analyses, facilitators spent the most time on the event of internal team meetings for problem solving and sharing updates, the target of executing, and the implementation strategy of facilitation. Additionally, events, targets, and strategies were identified for the most time-intensive outcomes - specifically, adoption, penetration, and fidelity.

This report describes a novel Implementation Log for quantitatively tracking a range of key implementation variables in near real time. Although modifications may improve end-user experience, the log has potential to improve the reporting, planning, and analysis of implementation efforts.