Leveraging Large Language Models to Address Common Vaccination Myths and Misconceptions

Area of Science:

• Artificial Intelligence in Healthcare
• Public Health Communication
• Vaccinology

Background:

• Public reliance on Large Language Models (LLMs) for health information is growing.
• The accuracy of LLMs in addressing vaccine myths and potential sycophantic behavior requires investigation.
• Vaccine misinformation poses a significant public health challenge.

Purpose of the Study:

• To evaluate the accuracy and clarity of LLM responses to common vaccine myths.
• To assess LLM performance under different user framings (skeptic vs. believer).
• To compare the myth-debunking capabilities of multiple LLM vendors.

Main Methods:

• Exploratory evaluation of three LLMs (GPT-5, Gemini 2.5 Flash, Claude Sonnet 4).
• Used official German vaccine myths and realistic user framings.
• Responses assessed by medical and marketing experts for accuracy, clarity, and misconception addressal.
• Readability analyzed using Flesch Reading Ease scores.

Main Results:

• All LLMs successfully refuted targeted vaccine misconceptions across all framings.
• High scientific accuracy (median 4.0-4.5) and clarity ratings were observed.
• Gemini 2.5 Flash and GPT-5 were ranked highest for lay communication clarity.
• Readability was generally low, particularly for the 'convinced believer' framing and Claude Sonnet 4.

Conclusions:

• General-purpose LLMs can generate accurate rebuttals to vaccine myths.
• Linguistic complexity and framing may hinder accessibility for the general public.
• Optimized LLM outputs could form the basis for public health myth-debunking tools, pending behavioral outcome studies.