Using a stakeholder-engaged approach to develop and validate electronic clinical quality measures.

Area of Science:

• Health Services Research
• Quality Improvement Science
• Dental Informatics

Background:

• Developing robust electronic clinical quality measures (eCQMs) is crucial for improving oral healthcare.
• Previous efforts may have lacked comprehensive stakeholder input, potentially impacting measure utility and adoption.

Purpose of the Study:

• To detail the stakeholder-engaged methodology for creating and validating two oral health eCQMs.
• To demonstrate a process for ensuring the feasibility, reliability, and validity of eCQMs.

Main Methods:

• Engaged diverse stakeholders throughout the measure development lifecycle, from conception to testing.
• Assessed data element feasibility via interviews and utilized synthetic patient datasets for implementation testing across electronic health record (EHR) systems.
• Validated measure implementation using automated EHR data reporting compared to manual record reviews, employing the kappa statistic.

Main Results:

• A dedicated stakeholder workgroup guided all development and testing phases.
• All critical data elements passed feasibility assessments, and measure implementation was proven feasible in EHRs.
• High reliability and validity were confirmed, with kappa statistics indicating "almost perfect" to "substantial" agreement, and the measures were published.

Conclusions:

• Stakeholder engagement is vital for successful eCQM development and testing.
• Early and continuous stakeholder involvement mitigates risks to feasibility, reliability, and validity, optimizing resource allocation.