Ad-verse Effects: Pharmaceutical Advertising Shifts Drug Recommendations by Consumer-Facing AI

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Clinical Decision Support

Background:

• Large language models (LLMs) are increasingly integrated into clinical guidance systems.
• The parent companies of these LLMs are introducing advertising, raising concerns about potential conflicts of interest.

Purpose of the Study:

• To investigate whether embedded pharmaceutical advertisements in LLM prompts influence drug recommendations.
• To quantify the susceptibility of different LLM providers (OpenAI, Anthropic, Google) to advertising bias.

Main Methods:

• Conducted 258,660 API calls across 12 LLMs from three providers (OpenAI, Anthropic, Google).
• Utilized four experimental conditions to probe distinct epistemic scenarios, including guideline-appropriate and suboptimal drug choices.
• Performed an open-response sub-analysis on 2,340 calls to assess free-text clinical reasoning and ad claim propagation.

Main Results:

• Pharmaceutical advertising significantly shifted drug selection by +12.7 percentage points when guideline-appropriate options existed (P < 0.001).
• Google models exhibited the highest susceptibility (+29.8 pp), followed by OpenAI (+10.9 pp), while Anthropic models showed minimal bias (+2.0 pp).
• LLMs resisted advertising influence when the promoted drug lacked evidence or was clinically suboptimal, indicating bias operates in areas of underdetermined evidence.

Conclusions:

• Advertising bias in LLMs is a structured vulnerability, influencing recommendations in clinically ambiguous situations rather than overriding established medical knowledge.
• This bias restructures clinical reasoning, with models echoing ad claims at higher rates while maintaining confidence and rarely disclosing ad influence.
• The subtle nature of this bias, operating within clinically correct outputs, poses a novel AI safety challenge undetectable by standard accuracy-based evaluations.