Breast Tumor Classification Based on MRI-US Images by Disentangling Modality Features

Area of Science:

• Medical imaging analysis
• Artificial intelligence in oncology
• Multimodal data fusion

Background:

• Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) and ultrasound (US) offer complementary information for breast tumor diagnosis.
• Existing machine learning methods often rely on single imaging modalities, limiting classification performance.
• Integrating multi-modal data for breast tumor classification remains a challenge.

Purpose of the Study:

• To develop a novel network, MUM-Net, for classifying breast tumors using paired 3D MRI and 2D US images.
• To investigate methods for effectively fusing multi-modality information to boost classification performance.
• To extract modality-agnostic features that are robust across different imaging techniques.

Main Methods:

• Proposed the MRI-US multi-modality network (MUM-Net) for breast tumor classification.
• Employed a discrimination-adaption module to decompose features into modality-agnostic and modality-specific components.
• Utilized a feature fusion module with an affinity matrix and nearest neighbor selection to enhance modality-agnostic features.
• Validated the method on a dataset of 502 paired MRI-US breast tumor cases.

Main Results:

• MUM-Net achieved high AUC scores in classifying lymph node metastasis (0.8581), histological grade (0.8965), and Ki-67 level (0.8577).
• The proposed method significantly outperformed single-modality and single-task approaches.
• Extracted modality-agnostic features demonstrated an ability to focus on relevant tumor regions in both MRI and US modalities.

Conclusions:

• MUM-Net effectively leverages paired multi-modality MRI and US data for enhanced breast tumor classification.
• The proposed feature decomposition and fusion strategy leads to improved diagnostic accuracy.
• Modality-agnostic features are crucial for robust tumor region identification across different imaging modalities.