Consensus and Complementary Feature Guided Multi-modal Knowledge Distillation Network for Breast Cancer Diagnosis

The integration of Hematoxylin and Eosin (H&E) and Immunohistochemistry (IHC) staining offers complementary information for accurate assessment of HER2 status in breast cancer. While multi-modal approaches enhance diagnostic precision, the high cost and acquisition constraints of IHC limit their routine clinical application. To address this, we propose a two-stage diagnostic framework that enables effective HER2 grading using only H&E images. In the first stage, a Consensus and Complementary Feature Co-Embedding Network (CoCoFNet) extracts modality-specific and cross-modal features to fully exploit multi-modal representations. In the second stage, a Hierarchical Multi-modal Knowledge Distillation (HM-KD) strategy transfers discriminative knowledge from the multi-modal teacher to a unimodal student network. Experiments on two public datasets demonstrate that the proposed method achieves comparable performance with state-of-the-art distillation methods using only H&E for HER2 grading. Furthermore, CoCoFNet demonstrates superior fusion capability, leading to more effective supervision and improved generalization in the unimodal student network. The codes can be publicly available from https://github.com/syguo95/HMKD.