The Cognitive Walkthrough for Implementation Strategies (CWIS): a pragmatic method for assessing implementation...

Area of Science:

• Implementation Science
• Human-Centered Design
• Mental Health Services Research

Background:

• Implementation strategies are crucial for integrating research into practice but often complex and impractical.
• A gap exists in methods to assess the usability and contextual fit of these strategies.
• Cognitive walkthroughs, adapted from human-centered design, offer a potential solution for evaluating strategy usability.

Purpose of the Study:

• To present a novel methodology, Cognitive Walkthrough for Implementation Strategies (CWIS), for evaluating the usability of implementation strategies.
• To apply CWIS to a post-training consultation strategy aimed at supporting school mental health clinicians in adopting measurement-based care.

Main Methods:

• CWIS is a six-step, mixed-methods approach including task analysis, scenario development, and group testing.
• Two group testing sessions were conducted with 10 clinician users.
• Participants provided qualitative justifications and quantitative ratings on task completion likelihood and overall strategy usability.

Main Results:

• Quantitative usability ratings for the consultation protocol averaged 71.3 (SD = 10.6), indicating low acceptability.
• Twenty-one usability problems were identified, including misalignments with clinical timelines and process digressions.
• Significant variability in anticipated success rates across participants and subtasks was observed.

Conclusions:

• The CWIS methodology effectively identified specific usability issues within the consultation strategy.
• The identified issues explained quantitative usability data and provided a roadmap for strategy enhancement.
• CWIS shows preliminary utility for assessing implementation strategy usability and informing redesign efforts.