Quantifying Emergency Medicine Residency Learning Curves Using Natural Language Processing: Retrospective Cohort...

Area of Science:

• Medical Education
• Emergency Medicine
• Computational Linguistics

Background:

• The optimal duration of emergency medicine (EM) residency training is debated, with potential standardization to 4 years.
• Limited empirical data exists on resident clinical exposure accumulation and diagnostic reasoning depth.
• Natural language processing (NLP) offers a novel method for comprehensive clinical experience quantification.

Purpose of the Study:

• Quantify clinical topic exposure acquisition in EM residents over time.
• Evaluate variations in exposure patterns among residents and graduating classes.
• Assess changes in workload and case complexity to inform optimal program length.

Main Methods:

• Retrospective cohort study of 62 EM residents and 244,255 ED encounters (2016-2023).
• Utilized a retrieval-augmented generation NLP pipeline to map clinical documentation to the 2022 Model for Clinical Practice of Emergency Medicine (MCPEM) subcategories.
• Analyzed cumulative topic exposure, coverage diversity, inter-resident variability, clinical complexity (ESI scores), and admission rates.

Main Results:

• Residents gained the most new topics in PGY1, with exposure plateauing around 39-41 months, but with significant individual variation.
• By PGY4, residents averaged 63.2% of MCPEM subcategories, a 9.9% increase over PGY3.
• Annual case volume more than tripled from PGY1 to PGY4, with increased case complexity (lower ESI scores, higher ESI 1-2 cases).

Conclusions:

• NLP provides a scalable, detailed method for tracking EM resident clinical exposure and progression.
• Residents continue to acquire new experiences, including higher-acuity cases, into their fourth year.
• A 4-year training model may offer additional educational value, emphasizing the need for individualized assessment due to learning trajectory variability.