SynthMedic: Utilizing large language models for synthetic discharge summary generation, correction and validation.

Area of Science:

• Medical Informatics
• Natural Language Processing
• Artificial Intelligence in Healthcare

Background:

• Synthetic clinical texts offer a privacy-preserving alternative to real patient data for training AI models.
• Current methods for generating synthetic medical data face challenges in ensuring factual accuracy and clinical relevance.
• The use of synthetic data can enhance transparency, reduce bias, and lower costs in medical AI development.

Purpose of the Study:

• To develop and validate a methodology for generating, validating, and correcting synthetic discharge summaries using LLMs.
• To create a high-quality, publicly available corpus of synthetic discharge summaries for training machine learning models.
• To ensure the medical factual correctness and clinical credibility of synthetic discharge summaries.

Main Methods:

• Utilized a large language model (LLM) to generate synthetic discharge summaries based on specific diseases and medical references (Merck Manuals).
• Employed both LLM-based and human expert validation to assess the quality and accuracy of the generated summaries.
• Implemented a Knowledge Graph-based approach for automatic correction to ensure medical factual accuracy.

Main Results:

• Human expert evaluation confirmed the credibility and factual accuracy of the synthetic discharge summaries when grounded with medical references.
• Achieved a System Usability Score of 94.35% from medical professionals and a 93.65% Faithfulness score from an LLM.
• The generated corpus includes 900 synthetic discharge summaries for nine significant diseases.

Conclusions:

• The proposed methodology effectively generates high-quality synthetic discharge summaries suitable for training AI models.
• The publicly available corpus and methodology empower the research community to develop advanced medical AI without using sensitive patient data.
• This work facilitates the development of AI tools to support healthcare professionals in their daily tasks.