Exposing Vulnerabilities in Clinical LLMs Through Data Poisoning Attacks: Case Study in Breast Cancer

Area of Science:

• Artificial Intelligence
• Biomedical Informatics
• Cybersecurity

Background:

• Large Language Models (LLMs) are increasingly used in healthcare, trained on vast datasets including public literature and clinical notes.
• Publicly accessible LLMs, like BioGPT, may inherit vulnerabilities from their training data, posing risks for sensitive applications.
• Data poisoning attacks, where malicious data is injected during training, represent a significant threat to LLM integrity.

Purpose of the Study:

• To investigate the susceptibility of clinical Large Language Models (LLMs) to data poisoning attacks.
• To assess the extent to which de-identified breast cancer clinical notes can be manipulated through such attacks.
• To highlight the urgent need for understanding and mitigating LLM vulnerabilities in the clinical domain.

Main Methods:

• The study focused on BioGPT, a clinical LLM trained on biomedical literature and MIMIC-III clinical notes.
• Researchers explored data poisoning attack vectors targeting de-identified breast cancer clinical notes.
• The impact of these attacks on the LLM's output was systematically evaluated.

Main Results:

• The research successfully demonstrated that clinical LLMs are vulnerable to data poisoning attacks.
• Manipulation of LLM outputs was achieved using poisoned de-identified breast cancer clinical notes.
• The findings underscore the potential for compromising sensitive clinical information and LLM-driven insights.

Conclusions:

• Clinical Large Language Models (LLMs) exhibit significant vulnerabilities to data poisoning, posing risks to patient data and diagnostic accuracy.
• There is a critical need for robust security measures and responsible deployment strategies for LLMs in healthcare settings.
• Further research into secure LLM training and validation is essential to ensure trustworthy AI in medicine.