Using Resident-Sensitive Quality Measures Derived From Electronic Health Record Data to Assess Residents' Performance...

Area of Science:

• Medical Education
• Health Informatics
• Pediatric Emergency Medicine

Background:

• Traditional quality metrics inadequately reflect resident clinical work and cannot link training to healthcare quality.
• Assessing resident performance is crucial for improving patient care and medical education outcomes.

Purpose of the Study:

• To determine if electronic health record (EHR) data can meaningfully assess resident clinical performance in pediatric emergency medicine.
• To evaluate the utility of resident-sensitive quality measures (RSQMs) derived from EHR data.

Main Methods:

• Analysis of EHR data for asthma and bronchiolitis RSQMs from a quaternary children's hospital (2017-2019).
• Residents were ranked using composite scores from raw, unadjusted, and case-mix adjusted latent score models.
• Reliability and associations between scoring models were compared; resident and patient characteristics were analyzed.

Main Results:

• Asthma data met reliability requirements (α = 0.77), showing high correlations between scoring models (r = 0.90-0.99).
• Bronchiolitis data did not meet reliability standards (α = 0.17).
• Case-mix adjustments shifted resident ranks by an average of 10 percentiles, with changes associated with resident experience and patient acuity.

Conclusions:

• EHR data, adjusted for patient complexity, can meaningfully assess resident clinical performance in specific areas.
• This approach identifies opportunities for quality improvement in pediatric residency training.