Retrieval augmented generation for large language models in healthcare: A systematic review

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Natural Language Processing

Background:

• Large Language Models (LLMs) show potential in healthcare but face limitations like outdated data, hallucinations, and lack of transparency.
• Retrieval Augmented Generation (RAG) addresses LLM limitations by integrating external knowledge sources.
• A systematic understanding of RAG applications, datasets, and evaluations in healthcare is currently lacking.

Purpose of the Study:

• To systematically review RAG-based approaches for LLMs in the healthcare domain.
• To analyze retrieval, augmentation, and generation strategies within healthcare RAG systems.
• To identify limitations, strengths, and research gaps in existing RAG literature for medical applications.

Main Methods:

• Comprehensive literature review of RAG methodologies applied to LLMs in healthcare.
• Analysis of datasets used, RAG techniques (Naive, Advanced, Modular), and LLM models (e.g., GPT-3.5/4).
• Assessment of evaluation frameworks and consideration of ethical implications in reviewed studies.

Main Results:

• A significant majority of reviewed studies (78.9%) utilize English datasets, with a smaller portion (21.1%) using Chinese datasets.
• Proprietary models like GPT-3.5/4 are predominantly used for RAG in healthcare.
• A notable lack of standardized evaluation frameworks and insufficient attention to ethical considerations in RAG for healthcare were identified.

Conclusions:

• RAG shows promise for improving LLM performance in healthcare, but current implementations lack standardization and ethical oversight.
• Further research is crucial to develop robust evaluation metrics and address ethical challenges for responsible AI adoption in clinical settings.
• Standardized frameworks and ethical guidelines are needed to ensure the safe and effective deployment of RAG-based LLMs in medicine.