A machine learning model using clinical notes to identify physician fatigue.

Area of Science:

• Medical Informatics
• Clinical Decision Making
• Artificial Intelligence in Healthcare

Background:

• Clinical notes are crucial for physician-patient encounters but may reflect physician fatigue.
• Physician fatigue can impact clinical decision-making and patient care.
• Identifying fatigue in clinical documentation is an unmet need.

Purpose of the Study:

• To develop and validate a model that identifies physician fatigue from clinical notes.
• To assess the impact of physician fatigue on clinical decision-making.
• To evaluate the detectability of fatigue in large language model-generated text.

Main Methods:

• Trained a predictive model on 129,228 emergency department (ED) visits to identify physicians working high-frequency shifts (≥5 in 7 days).
• Validated the model on a hold-out set, assessing its accuracy in identifying fatigued physicians and high-fatigue settings (overnight shifts, high patient volumes).
• Analyzed the correlation between model-predicted fatigue and the yield of diagnostic testing for myocardial infarction, and evaluated fatigue signals in large language model (LLM)-generated notes.

Main Results:

• The model accurately identified notes from high-workload physicians and flagged notes from high-fatigue settings.
• Increased model-predicted fatigue correlated with a 19% decrease in myocardial infarction testing yield per standard deviation.
• Notes generated by LLMs showed 74% higher predicted fatigue than physician-written notes, with increased word predictability.

Conclusions:

• Physician fatigue, identifiable through clinical note analysis, is associated with impaired clinical decision-making.
• The developed model can detect physician fatigue and its potential impact on patient care.
• LLMs may introduce subtle text characteristics indicative of fatigue, warranting further investigation into their reliability in clinical documentation.