DC-MSSFF Net: Dule-channel multi-scale spatial-spectral feature fusion network for cholangiocarcinoma pathology
Meiyan Liang1, Zelin Xi2, Bo Li3
1Shanxi Key Laboratory of Wireless Communication and Detection, School of Physics and Electronic Engineering, Shanxi University, Taiyuan, 030006, Shanxi, China; Shanxi Beike Biotechnology, Taiyuan, 030006, Shanxi, China.
Background And Objective:
High-precision segmentation of pathological images is a challenging task in the field of medical image processing. Hyperspectral microscopic imaging offers a distinct advantage in histopathological image segmentation due to its abundance of spectral and spatial data.
Methods:
Here, a Dule-Channel Multi-Scale Spatial-Spectral Feature Fusion Network (DC-MSSFF Net) is proposed for semantic segmentation of cholangiocarcinoma hyperspectral images (HSI). The DC-MSSFF Net is composed of two parallel channels, graph-within-graph (GwG) and multi-scale CNN. The GwG can greatly reduce the computational burden while establishing the spatial context relationship of the HSI image. The multi-scale CNN channel is able to fine-tune the segmented edges of the HSI images at the pixel-level based on hyperspectral information in the depth dimension. Afterwards, the segmentation results are achieved by fusing the features from the two channels. Furthermore, an ensemble-based framework is applied to further improve the performance of the model.
Results:
The image segmentation evaluation indexes such as dice similarity coefficient (Dice) of the Cholangiocarcinoma HSI data can reach 70.47, which is much higher than the SOTA method and RGB-based image segmentation methods.
Conclusion:
The superior performance of the DC-MSSFF network pioneers the inductive learning task of deep frameworks for semantic segmentation of high-resolution hyperspectral image (HR-HSI).