Enhancing vision-language model with pretraining for reasoning medical applications.

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Computer Vision

Background:

• Vision-Language Models (VLMs) are increasingly applied to medical tasks.
• Current VLMs often lack step-by-step reasoning, hindering complex medical information processing and clinical trust.
• There is a need for advanced AI models that can simulate clinical diagnostic reasoning.

Purpose of the Study:

• To develop a Multi-modal Medical Reasoning Model (MMRM) that integrates structured Chain of Thought (CoT) reasoning into VLMs.
• To enhance the diagnostic capabilities and clinical trustworthiness of AI in healthcare.
• To address the limitations of current VLMs in handling complex medical data.

Main Methods:

• Proposed an Ortho Enhanced Training Framework to optimize the VLM's visual encoder.
• Utilized black-box knowledge distillation to transfer medical CoT reasoning to the language model component.
• Constructed a novel multi-modal medical Chain of Thought dataset for explicit diagnostic reasoning training.

Main Results:

• Achieved state-of-the-art performance on medical Visual Question Answering (VQA) benchmarks like SLAKE.
• Demonstrated superior diagnostic accuracy and explanation quality compared to existing methods.
• The model provides interpretable reasoning pathways, enhancing clinical trustworthiness.

Conclusions:

• The MMRM effectively simulates clinical diagnosis by incorporating CoT reasoning.
• The model's ability to provide interpretable explanations strengthens clinical trust.
• This research paves the way for deploying AI-assisted healthcare solutions in real-world clinical settings.