MIC: Breast Cancer Multi-label Diagnostic Framework Based on Multi-modal Fusion Interaction.

Area of Science:

• Medical Imaging
• Artificial Intelligence in Medicine
• Oncology

Background:

• Automated diagnosis of breast ultrasound images is challenging due to low resolution and difficulty in recognition.
• Current multi-modal fusion methods often overlook textual pathology information and focus solely on binary classification (benign/malignant).
• Existing diagnostic approaches are insufficient for comprehensive clinical applications requiring detailed grading.

Purpose of the Study:

• To propose a novel multi-modal fusion interactive diagnostic framework (MIC) for breast cancer.
• To achieve multi-label classification, including benign-malignant diagnosis and Breast Imaging Reporting and Data System (BI-RADS) gradings (3, 4a, 4b, 4c, 5).
• To integrate brightness-mode ultrasound, contrast-enhanced ultrasound, and pathological information for enhanced diagnostic accuracy.

Main Methods:

• The MIC framework fuses brightness-mode ultrasound, contrast-enhanced ultrasound images, and pathological data.
• It employs a multi-modal similarity module to capture inter-modal high similarity features.
• An interactive feature enhancement module extracts global-local and multi-scale complementary information, while a cross-modal interaction module integrates pathology knowledge.

Main Results:

• The proposed framework achieved high performance metrics: 98.45% accuracy, 98.25% precision, 98.06% recall, and 98.43% F1-score.
• Experimental results demonstrate the framework's effectiveness in recognizing and classifying breast cancers.
• The multi-modal fusion approach significantly improved diagnostic capabilities compared to existing methods.

Conclusions:

• The MIC framework offers a robust solution for the automated, multi-label diagnosis of breast cancer.
• Integrating diverse data modalities, including pathology, enhances diagnostic precision and clinical utility.
• The framework effectively addresses the limitations of current methods in recognizing complex breast tumor characteristics.