CRITIC-RAG: Knowledge-Augmented Large Language Models With Verified Retrieval for Improved Medical Reasoning.

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Natural Language Processing

Background:

• Retrieval-augmented generation (RAG) improves large language models (LLMs) for question answering (QA).
• Challenges remain in factual consistency and reasoning reliability for RAG in knowledge-intensive domains like medicine.
• Enhancing trustworthiness in medical QA is crucial for clinical decision support.

Purpose of the Study:

• To propose CRITIC-RAG, a verification-enhanced framework to address RAG limitations in medical QA.
• To improve the factual consistency and reasoning reliability of LLM-based medical QA systems.
• To demonstrate the broad applicability and plug-and-play adaptability of the proposed framework.

Main Methods:

• Integrated a small-size, instruction-tuned verifier throughout the RAG pipeline.
• Employed selective retrieval, evidence filtering, structured reasoning via self-consistency, and groundedness verification.
• Conducted comprehensive experiments across five medical QA benchmarks and multiple LLM backbones.

Main Results:

• CRITIC-RAG significantly improved accuracy on MedQA (41.3% to 48.8%) and BERTScore on MedicationQA (68.1% to 74.4%) using LLaMA-3.
• Evidence filtering and structured reasoning were identified as critical components for robust performance through ablation studies.
• Verification stages jointly contributed to more accurate, evidence-grounded responses, as shown by case and retrieval analyses.

Conclusions:

• Verification is a key mechanism for enhancing trustworthiness in knowledge-intensive medical QA.
• CRITIC-RAG offers a practical solution for improving the reliability of LLMs in healthcare applications.
• The framework's adaptability makes it suitable for various LLM backbones and medical QA tasks.