Leveraging long context in retrieval augmented language models for medical question answering

Area of Science:

• Artificial Intelligence in Medicine
• Natural Language Processing
• Medical Informatics

Background:

• Large language models (LLMs) show promise for healthcare applications like medical literature summarization.
• LLMs face challenges with outdated knowledge and hallucination, limiting their use in evolving medical topics.
• Retrieval-augmented generation (RAG) enhances LLM accuracy by integrating external knowledge but can suffer from information retrieval issues like the "lost-in-the-middle" problem.

Purpose of the Study:

• To improve the robustness and reliability of the RAG workflow specifically for the medical domain.
• To address the "lost-in-the-middle" issue in RAG without altering LLM model weights.
• To enhance the safety and trustworthiness of LLM applications in critical healthcare tasks.

Main Methods:

• Proposed a novel map-reduce strategy named BriefContext.
• Implemented BriefContext to mitigate the "lost-in-the-middle" problem in RAG.
• Evaluated the workflow across various LLM backbones and medical question-answering datasets.

Main Results:

• The BriefContext strategy effectively combats the "lost-in-the-middle" issue in RAG.
• Demonstrated advantages of the proposed workflow with different LLM architectures.
• Validated the approach on multiple medical question-answering datasets, showing improved performance.

Conclusions:

• The BriefContext method enhances the reliability of RAG in the medical domain.
• This approach reduces misinformation risk and ensures retention of critical clinical information.
• Promises more trustworthy LLM deployment for medical question answering, clinical decision support, and patient-facing applications.