Socio-Demographic Modifiers Shape Large Language Models' Ethical Decisions.

Area of Science:

• Artificial Intelligence in Healthcare
• Medical Ethics
• Computational Linguistics

Background:

• The ethical alignment of large language models (LLMs) in clinical decision-making is not well understood.
• LLMs may exhibit biases influenced by socio-demographic factors, impacting healthcare equity.

Purpose of the Study:

• To investigate whether LLMs alter medical ethical decisions when presented with socio-demographic cues.
• To assess the susceptibility of various open-source LLMs to biases in clinical vignette evaluations.

Main Methods:

• Evaluated nine open-source LLMs using 100 clinical vignettes with yes/no ethical choices.
• Introduced socio-demographic modifiers to vignettes, repeating each scenario 10 times per model.
• Conducted approximately 0.5 million experiments to analyze LLM response shifts.

Main Results:

• All tested LLMs significantly changed responses based on socio-demographic details (p < 0.001).
• High-income cues promoted utilitarian choices, while marginalized-group cues increased autonomy considerations.
• No LLM maintained consistent ethical decision-making across all scenarios, with utilitarian choices showing the largest shifts.

Conclusions:

• Current LLMs can be unduly influenced by socio-demographic cues, posing risks to equitable healthcare.
• There is a critical need for auditing and alignment strategies to ensure LLM ethical behavior in clinical informatics.
• LLMs must be developed to respect ethical principles while accommodating clinical complexity and diversity.