Enhancing Medical Vision-Language Contrastive Learning via Inter-Matching Relation Modeling.

Area of Science:

• Artificial Intelligence
• Medical Imaging Analysis
• Computer Vision

Background:

• Medical vision-language contrastive learning (mVLCL) leverages medical imaging reports for supervision.
• Current mVLCL methods align image sub-regions with report keywords but ignore inter-matching relations.
• Existing methods aggregate local-matchings via simple pooling, failing to reason about semantic and importance relations.

Purpose of the Study:

• To propose a novel mVLCL method that models inter-matching relations for improved medical image representation learning.
• To introduce a relation-enhanced contrastive learning framework (RECLF) incorporating semantic and importance relation reasoning.
• To enhance fine-grained report supervision for medical vision tasks.

Main Methods:

• Developed a relation-enhanced contrastive learning framework (RECLF).
• Introduced a semantic-relation reasoning module (SRM) to model relationships between semantically related local-matchings.
• Incorporated an importance-relation reasoning module (IRM) to differentiate clinically important matchings.

Main Results:

• Evaluated RECLF on six public benchmark datasets across four downstream tasks (segmentation, zero-shot classification, linear classification, cross-modal retrieval).
• Demonstrated the superiority of RECLF over state-of-the-art mVLCL methods.
• Achieved consistent improvements in both single-modal and cross-modal tasks.

Conclusions:

• RECLF effectively models inter-matching relations, leading to improved medical image representations.
• The proposed method enhances generalization capabilities for downstream medical vision tasks.
• RECLF offers a more effective approach to medical vision-language contrastive learning.