Healthcare quality measures in implementation research: advantages, risks and lessons learned

Implementation studies evaluate strategies to move evidence-based practices into routine clinical practice. Often, implementation scientists use healthcare quality measures to evaluate the integration of an evidence-based clinical practice into real-world healthcare settings. Healthcare quality measures have standardized definitions and are a method to operationalize and monitor guideline-congruent care. Implementation scientists can access existing data on healthcare quality measures through various sources (e.g. operations-calculated), or they can calculate the measures directly from healthcare claims and administrative data (i.e. researcher-calculated). Implementation scientists need a better understanding of the advantages and disadvantages of these methods of obtaining healthcare quality data for designing, planning and executing an implementation study. The purpose of this paper is to describe the advantages, risks and lessons learned when using operations- versus researcher-calculated healthcare quality measures in site selection, implementation monitoring and implementation outcome evaluation. A key lesson learned was that relying solely on operations-calculated healthcare quality measures during an implementation study poses risks to site selection, accurate feedback on implementation progress to stakeholders, and the integrity of study results. A possible solution is using operations-calculated quality measures for monitoring of evidence-based practice uptake and researcher-calculated measures for site section and outcomes evaluation. This approach provides researchers greater control over the data and consistency of the measurement from site selection to outcomes evaluation while still retaining measures that are familiar and understood by key stakeholders whom implementation scientists need to engage in practice change efforts.