Spectral attribute reasoning for interpretable multi-modal pathological segmentation.

Area of Science:

• Computational pathology
• Digital pathology
• Medical image analysis

Background:

• Accurate segmentation of histological entities is crucial for computational pathology and clinical diagnosis.
• Microscopic hyperspectral images (MHSIs) and RGB images offer complementary pathological information.
• Existing multi-modal methods lack interpretability and fail to fully exploit spectral signatures in MHSIs.

Purpose of the Study:

• To develop an interpretable multi-modal segmentation network for MHSIs and RGB images.
• To address the limitations of existing methods in exploiting spectral signatures and providing explicit reasoning.
• To improve segmentation performance and interpretability in computational pathology.

Main Methods:

• Proposed Pisa-Net, a Pathology-Interpretable Spectral Attribute Learning Network for MHSI-RGB segmentation.
• Incorporated pathology knowledge via pathologist-selected spectral signatures for attribute learning.
• Utilized frequency-domain representation and adaptive re-weighting for feature extraction and fusion.
• Integrated RGB and MHSI features through sparse spatial compression guided by spectral evidence.

Main Results:

• Pisa-Net achieved superior segmentation performance for cells, glands, and tumors on public multi-modal pathology datasets.
• Demonstrated improved interpretability by explicitly linking predictions to pathology-aligned spectral evidence.
• Showcased effective multi-modal fusion consistent with pathological reasoning.

Conclusions:

• Pisa-Net offers a novel approach to interpretable multi-modal segmentation in computational pathology.
• The knowledge-driven spectral attribute learning enhances segmentation accuracy and clinical relevance.
• This method provides explicit, pathology-grounded spectral evidence for segmentation predictions.