M3-Net++: A multi-scale, multi-level, multi-stream network for nuclei segmentation in breast cancer histopathology
Arbab Sufyan Wadood1, Mohammad Faizal Ahmad Fauzi2, Wong Lai Kuan3
1Faculty of Artificial Intelligence and Engineering, Multimedia University, 63100, Cyberjaya, Malaysia; Department of Computer Science, Balochistan University of Information Technology, Engineering and Management Sciences (BUITEMS), Quetta, Pakistan.
Abstract:
Breast cancer remains a leading cause of morbidity and mortality worldwide. Histopathology, particularly the analysis of nuclear morphology in tissue samples, is critical for diagnosing and understanding the progression of breast cancer. Accurate nuclei segmentation plays a pivotal role in enabling detailed assessment of nuclear size, shape, and distribution patterns, which are essential for clinical diagnosis. However, traditional single-scale segmentation methods often fail to achieve this accuracy due to their inability to preserve fine details, capture broader contextual information, distinguish overlapping nuclei, and handle the inherent variability in nuclear morphology across different cell types. To address these challenges, we propose a multi-stream encoder-decoder architecture named M3-Net++ (Multi-Scale, Multi-Level, Multi-Stream Network), a novel deep learning model tailored for nuclei segmentation in histopathology images. M3-Net++ integrates both global and local tissue features for improved segmentation accuracy. The inclusion of Feature Refinement and Redundancy Elimination (FRRE) module further emphasizes critical features, while skip connections preserve high-resolution spatial information. To overcome challenges such as overlapping nuclei and class imbalance, we introduce a Hybrid Segmentation Loss (HSL) function. M3-Net++ achieves state-of-the-art performance on three benchmark datasets-ER-IHC, MoNuSAC, and CoNSeP-achieving Dice scores of 0.875, 0.871, and 0.872, and Panoptic Quality (PQ) scores of 0.730, 0.700, and 0.588, respectively, outperforming models such as HoVer-Net and SMILE. Despite these improvements, M3-Net++ remains computationally efficient, requiring only 46.18M parameters, 7.2 GB of peak GPU memory, and 0.119 s of inference time per 256×256 patch. These results highlight the robustness, adaptability, and clinical potential of M3-Net++ for breast cancer histopathology image analysis.
