MHD-Net: Memory-Aware Hetero-Modal Distillation Network for Thymic Epithelial Tumor Typing With Missing Pathology...

Area of Science:

• Medical Imaging
• Computational Pathology
• Artificial Intelligence in Oncology

Background:

• Accurate tumor typing is crucial for effective cancer treatment.
• Multi-modal data fusion (radiology and pathology) enhances diagnostic accuracy.
• Pathology data collection is costly and often post-surgical, limiting real-time diagnostic applications.

Purpose of the Study:

• To develop a method for learning from multi-modal data during training while relying solely on radiology data during testing.
• To improve tumor typing accuracy in scenarios with missing pathology data.
• To propose a novel Memory-aware Hetero-modal Distillation Network (MHD-Net).

Main Methods:

• Proposed Memory-aware Hetero-modal Distillation Network (MHD-Net) with teacher-student architecture.
• Developed a spatial-differentiated hetero-modal fusion module (SHFM) for teacher model.
• Introduced a contrast-boosted typing memory module (CTMM) for pathology feature storage.
• Implemented a multi-stage memory-aware distillation (MMD) scheme for the student model.
• Created a Radiology-Pathology Thymic Epithelial Tumor (RPTET) dataset.

Main Results:

• MHD-Net significantly improves tumor typing accuracy.
• The proposed method outperforms existing multi-modal approaches in missing modality scenarios.
• Experiments validated on RPTET and CPTAC-LUAD datasets.

Conclusions:

• MHD-Net effectively distills multi-modal knowledge for robust tumor typing.
• The approach addresses the challenge of missing pathology data in clinical settings.
• This method offers a promising solution for enhancing diagnostic accuracy in oncology.