Quantifying Hallucinations in Language Language Models on Medical Textbooks.

Area of Science:

• Natural Language Processing
• Artificial Intelligence in Medicine
• Medical Question Answering

Background:

• Hallucinations, or factually incorrect claims by large language models (LLMs), pose a significant challenge in natural language processing.
• Current medical question answering (QA) benchmarks seldom assess LLM hallucination against a fixed evidence source.

Purpose of the Study:

• To quantify hallucination prevalence in textbook-grounded medical QA.
• To compare hallucination rates and clinician preferences across different LLMs.

Main Methods:

• Experiment 1: Assessed hallucination frequency of LLaMA-70B-Instruct on novel medical QA prompts with provided passages.
• Experiment 2: Evaluated hallucination rates and clinician preference for responses from multiple LLMs.
• Clinician agreement was measured using quadratic weighted kappa and Kendall's tau-b.

Main Results:

• LLaMA-70B-Instruct exhibited a 19.7% hallucination rate in experiment one, despite 98.8% of responses being deemed highly plausible.
• Experiment two showed a negative correlation between hallucination rates and clinician usefulness scores (ρ=-0.71, p=0.058).
• High inter-rater reliability was observed among clinicians.

Conclusions:

• LLMs demonstrate a notable tendency to hallucinate in medical QA tasks, even when responses appear plausible.
• Reducing hallucination rates in LLMs is crucial for improving their utility and trustworthiness in clinical settings.
• Further research is needed to develop effective mitigation strategies for LLM hallucinations.