Local Alignment for Medical Vision-Language Pre-Training.

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Natural Language Processing

Background:

• Establishing semantic correspondences between medical images and reports is vital for vision-language pre-training.
• Existing methods struggle with small/blurry lesion regions and noisy medical reports, hindering accurate alignment.

Purpose of the Study:

• To introduce MedAligner, a novel network designed to enhance local semantic alignment in medical vision-language pre-training.
• To address challenges in lesion localization and semantic alignment caused by image characteristics and report redundancy.

Main Methods:

• MedAligner utilizes dual encoders for global and local representations, employing global contrastive learning for semantic consistency.
• Introduces Word-Region Alignment with a learnable, sparsified word-pixel similarity matrix for accurate lesion identification.
• Incorporates Diagnostic Term Filtering to focus on high-importance terms and aligns them with lesions using local contrastive loss.
• Employs a progressive training strategy involving report reconstruction and dynamic similarity updates for refined image-text pairs.

Main Results:

• MedAligner significantly outperforms existing methods in phrase grounding, image-text retrieval, and zero-shot classification.
• Demonstrates superior performance in establishing accurate local semantic correspondences between medical images and reports.
• Sets new benchmarks for medical vision-language pre-training tasks.

Conclusions:

• MedAligner effectively overcomes limitations in current medical vision-language pre-training approaches.
• The proposed methods enhance the precision of lesion localization and semantic alignment.
• This work advances the field of medical AI by improving the integration of visual and textual data.