Medical triage as an AI ethics benchmark.

Area of Science:

• Medical Ethics
• Artificial Intelligence
• Machine Learning

Background:

• Evaluating AI's ethical decision-making is crucial, especially in high-stakes medical scenarios.
• Existing benchmarks may not fully capture the complexities of real-world ethical dilemmas.
• Mass casualty incidents present unique ethical challenges for AI systems.

Purpose of the Study:

• To introduce the TRIAGE benchmark for assessing AI ethical decision-making in mass casualty events.
• To evaluate the performance of major large language models (LLMs) on medical ethical dilemmas.
• To investigate the impact of ethical and adversarial prompts on LLM behavior.

Main Methods:

• Development of the TRIAGE benchmark using medical dilemmas from healthcare professionals.
• Evaluation of six prominent LLMs using the TRIAGE benchmark.
• Analysis of performance variations based on different prompting strategies (ethical principles, adversarial prompts).

Main Results:

• Most LLMs exceeded random guessing but showed significant ethical errors, particularly open-source models.
• Providing explicit ethical principles to LLMs decreased their performance on TRIAGE.
• Adversarial prompts substantially reduced the accuracy of LLM ethical decision-making.

Conclusions:

• LLM performance in medical ethics is highly sensitive to context and prompt framing.
• Current AI systems have limitations in high-stakes ethical decision-making within medicine.
• Further research is needed to improve AI's ethical reasoning in critical healthcare situations.