Inherent Bias in Large Language Models: A Random Sampling Analysis.

Area of Science:

• Medical Ethics and Artificial Intelligence
• Health Informatics and Bias Detection

Background:

• Growing concerns exist regarding inherent bias, safety, and misinformation potential of large language models (LLMs).
• These concerns have significant implications for the integration of AI in healthcare decision-making.

Purpose of the Study:

• To investigate whether generative artificial intelligence (AI)-based simulations of physicians exhibit bias in life-and-death decisions.
• To assess bias in resource-scarce clinical scenarios using AI simulations.

Main Methods:

• Developed 13 questions simulating physicians in resource-limited environments making critical treatment choices.
• Utilized OpenAI's GPT-4 to simulate 1000 unique physicians and patients per question, ensuring diverse demographics.
• Patients had similar a priori survival likelihoods; physicians chose one patient to save based on limited resources.

Main Results:

• Simulated physicians consistently demonstrated racial, gender, age, political affiliation, and sexual orientation bias.
• Physicians predominantly favored patients sharing their own demographic characteristics (P<.05).
• Specific biases observed included nondescript physicians favoring White, male, young patients; political affiliation influenced choices (Democrats favored Black/female; Republicans favored White/male).

Conclusions:

• Publicly available large language models exhibit significant biases in simulated clinical decision-making.
• These biases could negatively impact patient outcomes if AI tools are used in clinical support without safeguards.
• Urgent need for bias mitigation strategies in AI for healthcare applications.