Designing for implementation: development, delivery, and multi-method evaluation of a clinician training program for...

Implementation strategies like user training are often excluded from research activities for digital health technologies (DHTs) or reserved for advanced stages of evidence generation. This may contribute to challenges in clinical translation due to usability, feasibility, fidelity, and scalability issues. Using the example of a concussion assessment tool called R2Play, we demonstrate how clinicians can be engaged in developing training resources for DHTs during early stages of evidence generation and illustrate potential benefits of this approach.

(1) Develop a clinician training program for R2Play administration; (2) Deliver the R2Play training program to clinician participants across multiple research sites; (3) Evaluate the training program based on participant reactions and learning using a multi-method approach.

The Kern Model for Medical Curricula guided training development, which involved a needs assessment, specifying learning objectives, and selecting education strategies. Training was delivered to clinician participants at four sites in two parts: (1) asynchronous online learning modules (1 h); and (2) live training session to practice R2Play administration tasks (1 h). Evaluation measures based on the Kirkpatrick Model for Training Evaluation included module-embedded knowledge check questions, live training task completion scores, post-training survey, and follow-up interview. Quantitative measures were analyzed descriptively (frequencies, proportions, means), and content analysis was applied to interview data.

Eighteen clinicians participated (n = 15 women), primarily physiotherapists (n = 11) and athletic therapists (n = 3). Survey and interview feedback generally indicated positive reactions (satisfaction, relevance, engagement) and learning perceptions (knowledge, skills, confidence). Participants objectively and subjectively experienced most difficulty in learning system set-up, delivering precise assessment level instructions, and explaining assessment scoring metrics. Interview data highlighted topics requiring additional content (navigating software interface, interpreting results, troubleshooting). Suggested improvements included an end-to-end workflow demonstration, mock run-through, and opportunities to observe experienced users.

Training programs for DHTs should consider addressing technical proficiency (i.e., system set-up and operation), clinical reasoning (i.e., interpreting data), and communication with care recipients. Early multicenter studies involving end-user training can help enhance the usability of DHTs and identify additional strategies to foster implementation. This work adds to limited empirical literature on how to prepare clinicians for using DHTs through evidence-based practical recommendations.