Key concept learning for medical vision language model with reasoning capabilities.

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Computer Vision

Background:

• Training medical vision-language models (VLMs) requires vast datasets, posing data acquisition challenges.
• General-domain VLMs lack specialized medical knowledge and reasoning capabilities.

Purpose of the Study:

• To introduce ConceptVLM, a data-efficient fine-tuning paradigm for medical VLMs.
• To enable transformation of general VLMs into specialized medical models with minimal labeled data.

Main Methods:

• ConceptVLM employs a key concept-aware training strategy.
• A structured medical concept dictionary and masked attention focus on essential clinical concepts.
• Fine-tuning preserves general capabilities while enhancing domain-specific comprehension.

Main Results:

• ConceptVLM achieved state-of-the-art performance on multimodal medical benchmarks.
• The model utilized only 1% of the original training data compared to traditional methods.
• Outperformed traditional methods reliant on large-scale question-answering datasets.

Conclusions:

• Concept-guided tuning is an effective strategy for developing cognitively capable medical VLMs.
• Challenges the necessity of extensive annotated corpora for medical VLM training.
• Demonstrates a viable path for data-efficient specialization of VLMs in the medical domain.