LLM-Guided Pain Management: Examining Socio-Demographic Gaps in Cancer vs non-Cancer cases.

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Health Equity Research

Background:

• Large language models (LLMs) present potential for clinical applications but carry risks of embedded socio-demographic biases.
• Opioid prescribing decisions are critical, requiring careful balancing of pain management and addiction risk, particularly during the ongoing opioid epidemic.

Purpose of the Study:

• To evaluate socio-demographic biases in large language model (LLM) recommendations for acute pain management.
• To assess how cancer diagnosis influences disparities in LLM outputs across various demographic groups.

Main Methods:

• Ten LLMs (open- and closed-source) were tested on 1,000 acute-pain vignettes, with half labeled as cancer and half as non-cancer.
• Vignettes were presented with 34 socio-demographic variations, analyzing recommendations for opioids, anxiety treatment, risk scores, and monitoring.
• Logistic and linear mixed-effects models were used to quantify output variations by demographic group and cancer status.

Main Results:

• Historically marginalized groups (unhoused, Black, LGBTQIA+) often received disproportionately higher or stronger opioid recommendations, exceeding 90% in cancer cases.
• Elevated risk scores were assigned to low-income or unemployed groups, yet they received fewer opioid recommendations, indicating inconsistent model logic.
• Disparities in anxiety treatment and perceived psychological stress recommendations were concentrated within marginalized populations, irrespective of identical clinical details.

Conclusions:

• LLM outputs demonstrate significant model-driven bias in pain management recommendations, diverging from clinical guidelines.
• Rigorous bias evaluation and the incorporation of guideline-based checks are essential for LLMs to ensure equitable and evidence-based clinical care.