Model confrontation and collaboration: A debate intelligence framework for enhancing medical reasoning in large...

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems
• Natural Language Processing for Healthcare

Background:

• Medical reasoning is crucial for clinical decision-making, including diagnosis and treatment planning.
• Peer interaction in psychology enhances self-correction, suggesting potential for AI model improvement.
• Existing ensemble methods for AI lack iterative refinement and self-reflection capabilities.

Purpose of the Study:

• To introduce Model Confrontation and Collaboration (MCC), a novel debate intelligence framework for AI.
• To leverage structured multi-round debate among diverse Large Language Models (LLMs) for enhanced medical reasoning.
• To improve accuracy and performance in medical question answering and diagnostic tasks.

Main Methods:

• Developed MCC, a framework integrating critique and self-reflection for iterative LLM reasoning refinement.
• Implemented structured, multi-round confrontation and collaboration among diverse LLMs.
• Evaluated MCC on multiple-choice benchmarks (MedQA, PubMedQA, MMLU medical subsets) and long-form medical QA.

Main Results:

• MCC achieved high accuracy on MedQA (92.6%) and PubMedQA (84.8%), outperforming individual LLMs.
• In long-form QA, MCC surpassed all individual LLMs and Med-PaLM 2 in physician and layperson evaluations.
• MCC demonstrated superior performance in diagnostic dialogue, achieving an 80% top-1 diagnosis rate.

Conclusions:

• MCC is a scalable, model-agnostic framework that significantly advances medical reasoning capabilities.
• Collaborative deliberation among LLMs via MCC enhances accuracy in medical question answering and diagnosis.
• The framework shows promise for improving AI-driven clinical decision support.