Developing Resident-Sensitive Quality Measures for Internal Medicine

Area of Science:

• Medical Education
• Health Services Research
• Quality Improvement

Background:

• Residency education aims to train physicians for high-quality patient care.
• Residents often lack data-driven feedback due to a scarcity of resident-level quality measures.
• Electronic health records (EHRs) offer a potential data source for quality assessment.

Purpose of the Study:

• To develop and evaluate resident-sensitive quality measures (RSQMs) using EHR data.
• To inform graduate medical education (GME) through objective quality metrics.
• To assess the feasibility and utility of RSQMs in internal medicine residency.

Main Methods:

• A cohort study involving senior internal medicine residents and patient admissions across 5 Canadian teaching hospitals (2010-2019).
• Development of 7 RSQMs based on clinical practice guidelines for pneumonia and general care.
• Application of a care variation framework (low value, discretionary, evidence-based) to categorize RSQMs.

Main Results:

• The study included 132,291 patient admissions linked to 793 residents.
• Low-value care measures (e.g., inappropriate red cell transfusions) showed minimal variation.
• Discretionary measures (e.g., antibiotic use, advanced imaging) exhibited significant resident-level variation.
• An evidence-based measure for first-line pneumonia antibiotic use demonstrated sensitivity to data extraction time windows.

Conclusions:

• An approach for developing and evaluating RSQMs using EHR data was established.
• RSQMs have potential for evaluating internal medicine residents' quality of care.
• Program-level application of RSQMs may be more appropriate than individual resident assessment due to attribution and statistical reliability concerns.