Advancing radiology foundation models with reasoning through step-by-step verification from daily reports

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Clinical Decision Support Systems

Background:

• Large language models (LLMs) and multimodal LLMs (MLLMs) show promise in complex tasks.
• Existing medical AI models often neglect structured clinical reasoning processes.

Purpose of the Study:

• To develop a radiology diagnosis MLLM, ChestX-Reasoner, that integrates explicit step-by-step reasoning.
• To create a benchmark and metric for evaluating the reasoning capabilities of medical AI.

Main Methods:

• A two-stage pipeline was used to train ChestX-Reasoner with process supervision from clinical reports.
• Introduced RadRBench-CXR, a benchmark with 59K visual question answering samples and 301K reasoning steps.
• Developed RadRScore to evaluate the factuality, completeness, and effectiveness of reasoning.

Main Results:

• ChestX-Reasoner demonstrated a 16% improvement in reasoning ability over leading medical models and 8.5% over general-purpose models.
• Diagnostic accuracy saw improvements ranging from 3.3% to 24% compared to baselines.
• The model's reasoning capabilities were significantly enhanced.

Conclusions:

• Incorporating explicit reasoning steps enhances diagnostic outcomes in medical AI.
• Process supervision improves the reliability and transparency of AI-driven medical diagnosis.
• ChestX-Reasoner represents a significant advancement in medical AI reasoning.