Visualizing Radiologic Connections: An Explainable Coarse-to-Fine Foundation Model with Multiview Mammograms and...

Area of Science:

• Artificial Intelligence in Medical Imaging
• Digital Mammography Analysis
• Computer-Aided Breast Cancer Detection

Background:

• Digital mammography generates complex data from images and reports.
• Label-limited settings pose challenges for training robust AI models.
• Extracting fine-grained visual-language representations is crucial for accurate analysis.

Purpose of the Study:

• To develop a foundational pretraining method for digital mammography.
• To extract fine-grained visual-language representations from mammograms and reports.
• To improve automated breast cancer analysis in label-limited scenarios.

Main Methods:

• Developed a multiview mammogram-report pretraining framework for automated breast cancer analysis.
• Incorporated an abnormality-aware technique tailored to dense fibroglandular tissue.
• Evaluated the framework on malignancy classification, segmentation, and localization tasks across four external medical centers.

Main Results:

• The model successfully captured relationships between multiview mammograms and reports, as shown by visualization results.
• Reduced false positives for breast cancer by 37% and enabled zero-shot abnormality segmentation.
• Outperformed existing approaches in malignancy classification (e.g., INbreast AUC: 0.90 vs 0.78) and segmentation/localization (e.g., INbreast Dice: 0.75 vs 0.63).

Conclusions:

• The proposed framework enhances interpretability in digital mammography.
• It enables fine-grained multimodal foundational learning for multiview mammograms and reports.
• This approach shows significant potential for improving AI-driven breast cancer diagnostics.