Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models.

Area of Science:

• Artificial Intelligence
• Medical Imaging Analysis
• Computer Vision

Background:

• Vision-language models (VLMs) excel in natural image tasks but are underexplored in medical imaging.
• Medical vision-language tasks require precise understanding and clinically relevant answers, hindered by data complexity and annotation scarcity.
• Traditional supervised fine-tuning (SFT) and Chain-of-Thought (CoT) methods are limited in medical VLM applications.

Purpose of the Study:

• To develop a reinforcement learning (RL)-enhanced VLM, Med-R1, for improved generalization and reliability in medical reasoning.
• To address the limitations of existing methods in handling complex medical data and scarce annotations.
• To investigate the impact of reasoning strategies on medical visual question answering (VQA) performance and interpretability.

Main Methods:

• Proposed Med-R1, a reinforcement learning-enhanced VLM utilizing Group Relative Policy Optimization (GRPO) for reward-guided learning beyond static annotations.
• Evaluated Med-R1 across eight diverse medical imaging modalities and five question types to assess performance and cross-task generalization.
• Explored different reasoning strategies, including omitting intermediate rationales (No-Thinking Med-R1) and generating rationales after initial answers (Think-After Med-R1).

Main Results:

• Med-R1 achieved a 29.94% average accuracy improvement over its base model (Qwen2-VL-2B) and outperformed a larger model (Qwen2-VL-72B).
• Demonstrated a 32.06% improvement in question-type generalization compared to Qwen2-VL-2B, also surpassing Qwen2-VL-72B.
• Found that omitting intermediate rationales improved cross-domain generalization, while the Think-After variant enhanced performance and interpretability, challenging assumptions about reasoning's role.

Conclusions:

• Reinforcement learning, specifically GRPO, effectively enhances VLM performance in medical imaging tasks.
• The quality and placement of reasoning steps, not merely their presence, are critical for effective medical VQA.
• Med-R1 offers a promising direction for developing more reliable and generalizable AI tools for medical image analysis.