Real-World Performance Measurement of Patient-Centered Clinical Decision Support Tools: Qualitative Study

Patient-centered clinical decision support (PC CDS) includes digital technology designed to give patients, caregivers, and clinicians evidence-based, patient-specific clinical guidance to inform care decisions. PC CDS interventions cover a range of use cases, but gaps in measurement make it difficult to assess the impact of these technologies on patient and clinician decision-making, care processes, and outcomes.

This study aimed to identify common measurement areas used in PC CDS projects, as well as measurement gaps, and to develop recommendations for future advancement of PC CDS measurement.

We conducted a systematic review of 20 exemplary PC CDS projects funded by the Agency for Healthcare Research and Quality's Digital Healthcare Research Program using predefined inclusion criteria. We reviewed published project materials (n=40) to gather information on the type of data, technology, and measures used, as well as conditions and populations addressed. Next, we identified a purposive sample of 9 projects and conducted key informant interviews with the principal investigators to gather perspectives on PC CDS performance measures and measurement-related limitations and challenges. We conducted a qualitative thematic synthesis to identify key themes from the reviewed material and interviews related to commonly used measures and measurement gaps in relation to a PC CDS performance measurement framework. We also gathered feedback on findings from a seven-member technical expert committee.

Overall, usability was the most common area of measurement across PC CDS design, development, implementation, and use phases. Projects focusing on designing and developing PC CDS technology also frequently measured acceptability, while projects focusing on implementation and evaluation frequently measured patient health outcomes, patient engagement, and clinician performance. Informants reported challenges with measuring the safety, timeliness, and cost of PC CDS technology. They also expressed the need for new measurement approaches to capture long-term health outcomes, benchmarks for meaningful patient engagement, and perspectives from people with limited digital or health literacy.

Project investigators are using numerous measures to assess real-world PC CDS interventions, yet there are several measurement areas that need more development. The findings revealed six important next steps for future development of PC CDS performance measurement that can significantly advance the use of technologies that promote patient-centered care: (1) tracking technical performance postimplementation, (2) evaluating interventions in lower-resourced health care settings, (3) conducting more cost assessments to inform scalability, (4) streamlining privacy and security management for external data storage, (5) refining and developing standardized patient engagement measures, and (6) adapting measurement tools to populations with limited English proficiency or digital and/or health literacy.