Considerations for Patient Privacy of Large Language Models in Health Care: Scoping Review

Area of Science:

• Healthcare AI
• Medical Informatics
• Data Privacy

Background:

• Large Language Models (LLMs) offer significant potential in healthcare for patient care and research.
• Protecting patient health information (PHI) is a critical challenge in the application of LLMs within healthcare.

Purpose of the Study:

• To evaluate current approaches for patient privacy protection in healthcare studies utilizing PHI and LLMs.
• To identify areas for improvement in safeguarding PHI when developing and applying LLMs in healthcare.

Main Methods:

• A scoping review of literature published between January 1, 2022, and July 20, 2025, was conducted.
• Searches were performed on PubMed and Embase, focusing on LLM development and application with PHI in healthcare.
• Included studies focused on LLM construction, fine-tuning, optimization, testing, and performance comparison using PHI.

Main Results:

• Out of 464 included studies, 9.7% neglected ethical review and 31.9% lacked patient informed consent.
• 38.4% of studies failed to report effective PHI protection measures.
• A significant lack of transparency and detail was observed in anonymization and deidentification methods.

Conclusions:

• Comprehensive recommendations for study design, implementation, and reporting are proposed to enhance privacy and transparency in PHI-LLM research.
• There is an urgent need for stricter regulatory frameworks and advanced privacy technologies to protect PHI.
• Future healthcare LLM applications must balance innovation with robust patient privacy protection for ethical and credible advancements.