Applying cognitive walkthrough methodology to improve the usability of an equity-focused implementation strategy

Our research team partnered with primary care and quality improvement staff in Federally Qualified Community Health Centers (CHCs) to develop Partnered and Equity Data-Driven Implementation (PEDDI) to promote equitable implementation of evidence-based interventions. The current study used a human-centered design methodology to evaluate the usability of PEDDI and generate redesign solutions to address usability issues in the context of a cancer screening intervention.

We applied the Cognitive Walkthrough for Implementation Strategies (CWIS), a pragmatic assessment method with steps that include group testing with end users to identify and prioritize usability problems. We conducted three facilitated 60-min CWIS sessions with end users (N = 7) from four CHCs that included scenarios and related tasks for implementing a colorectal cancer (CRC) screening intervention. Participants rated the likelihood of completing each task and identified usability issues and generated ideas for redesign solutions during audio-recorded CWIS sessions. Participants completed a pre-post survey of PEDDI usability. Our research team used consensus coding to synthesize usability problems and redesign solutions from transcribed CWIS sessions.

Usability ratings (scale 0-100: higher scores indicating higher usability) of PEDDI averaged 66.3 (SD = 12.4) prior to the CWIS sessions. Scores averaged 77.8 (SD = 9.1) following the three CWIS sessions improving usability ratings from "marginal acceptability" to "acceptable". Ten usability problems were identified across four PEDDI tasks, comprised of 2-3 types of usability problems per task. CWIS participants suggested redesign solutions that included making data fields for social determinants of health and key background variables for identifying health equity targets mandatory in the electronic health record and using asynchronous communication tools to elicit ideas from staff for adaptations.

Usability ratings indicated PEDDI was in the acceptable range following CWIS sessions. Staff identified usability problems and redesign solutions that provide direction for future improvements in PEDDI. In addition, this study highlights opportunities to use the CWIS methodology to address inequities in the implementation of cancer screening and other clinical innovations in resource-constrained healthcare settings.