MedARC: Adaptive multi-agent refinement and collaboration for enhanced medical reasoning in large language models

Area of Science:

• Artificial Intelligence
• Biomedical Informatics
• Natural Language Processing

Background:

• Large Language Models (LLMs) show promise in medical question answering (QA).
• Clinical deployment is hindered by LLM limitations like hallucinations and inconsistent reasoning.
• Handling complex biomedical information remains a challenge for current LLMs.

Purpose of the Study:

• To introduce MedARC (Medical Agent Refinement and Collaboration), a multi-agent framework for enhanced medical QA.
• To address limitations of LLMs in medical QA through structured inter-agent debate.
• To improve factual consistency, completeness, and reliability of LLM-generated medical answers.

Main Methods:

• MedARC employs a multi-agent framework with structured debate among LLM agents.
• Key mechanisms include structured inter-agent summarization for refining agreements/disagreements.
• Confidence-aware aggregation synthesizes final answers from reliable agent contributions.

Main Results:

• MedARC significantly improves performance on medical QA benchmarks compared to zero-shot and CoT baselines.
• Accuracy increased from 72.9% to 77.2% on PubMedQA using DeepSeek-V3.
• Human evaluations confirm enhanced factual consistency and completeness with MedARC.

Conclusions:

• MedARC offers a reliable and scalable solution for LLM-based medical QA.
• The framework effectively addresses LLM limitations in clinical settings.
• Open-source code is available for further research and development.