From Promising Capabilities to Pervasive Bias: Assessing Large Language Models for Emergency Department Triage

Area of Science:

• Artificial Intelligence in Healthcare
• Clinical Decision Support Systems
• Health Equity Research

Background:

• Large Language Models (LLMs) demonstrate potential for clinical applications.
• The use of LLMs for emergency department (ED) triage is not well-studied.
• Assessing LLM performance requires evaluating robustness and identifying biases.

Purpose of the Study:

• To systematically investigate Large Language Model (LLM) capabilities in emergency department triage.
• To evaluate LLM robustness against distribution shifts and missing data.
• To identify intersectional biases in LLMs related to sex and race.

Main Methods:

• Comparison of various LLM approaches (continued pre-training, in-context learning) with conventional machine learning (ML).
• Assessment of robustness to distribution shifts and missing data.
• Counterfactual analysis to audit intersectional biases across sex and race.

Main Results:

• LLMs demonstrated superior robustness compared to traditional ML models.
• LLM methods leveraging similar past patient cases were most effective for triage.
• Reasoning capabilities of LLMs provided minimal benefit for triage tasks.
• Significant sex-based differences and pronounced racial disparities were identified in LLM performance.

Conclusions:

• LLMs offer enhanced robustness for ED triage, with case-based similarity being a key factor.
• LLMs exhibit critical intersectional biases related to sex and race, requiring careful auditing.
• Counterfactual analysis is a valuable systematic method for identifying pre-integration biases in LLMs.