Geometry-Guided Local Alignment for Multi-View Visual Language Pre-Training in Mammography

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Mammography screening is crucial for early breast cancer detection.
• Deep learning (DL) shows promise for improving mammography interpretation speed and accuracy.
• Current visual language models (VLMs) struggle with medical images due to data limitations and domain differences, often ignoring critical multi-view relationships in mammography.

Purpose of the Study:

• To address the limitations of existing VLMs in mammography by developing a model that properly incorporates multi-view correspondence.
• To improve the accuracy and efficiency of mammography interpretation using deep learning.

Main Methods:

• We propose GLAM (Global and Local Alignment for Multi-view mammography), a VLM pretraining method using geometry guidance.
• GLAM leverages prior knowledge of mammogram imaging to learn local cross-view alignments and fine-grained features.
• The model employs joint global and local, visual-visual, and visual-language contrastive learning.

Main Results:

• GLAM was pretrained on the EMBED dataset, one of the largest open mammography datasets.
• The proposed model demonstrated superior performance compared to baseline methods across multiple datasets and settings.
• GLAM effectively models multi-view correspondence, capturing critical geometric context often missed by other approaches.

Conclusions:

• GLAM offers a significant advancement in VLM pretraining for mammography by incorporating multi-view geometric relationships.
• This approach enhances the model's ability to interpret mammograms, leading to improved diagnostic accuracy.
• GLAM provides a foundation for more robust and accurate AI-driven breast cancer screening tools.