Segmentation ability map: Interpret deep features for medical image segmentation.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Deep convolutional neural networks (CNNs) are prevalent in medical image segmentation.
• Typically, only the output layer of CNNs is utilized, leaving deep feature representations underexplored.
• Understanding these hidden features is crucial for improving segmentation accuracy and model interpretability.

Purpose of the Study:

• To introduce Prototype Segmentation (ProtoSeg), a method for computing binary segmentation maps from deep features.
• To quantify the segmentation capabilities of deep features using a Segmentation Ability (SA) score.
• To enhance the interpretability and explainability of deep neural networks in medical image segmentation.

Main Methods:

• ProtoSeg computes segmentation maps directly from deep learned features.
• The Segmentation Ability (SA) score is calculated by comparing feature-derived segmentation maps with ground-truth using the Dice coefficient.
• A mean SA score is derived for performance estimation on test images without ground-truth.

Main Results:

• The SA score effectively quantifies the segmentation abilities of deep features across different layers and units.
• ProtoSeg provides insights into the segmentation capacity of individual input images.
• The method was successfully applied to diverse medical imaging datasets, including brain MRI, skin lesions, CT scans for COVID-related abnormalities, abdominal MRI for prostate segmentation, and CT for pancreatic mass segmentation.

Conclusions:

• ProtoSeg offers a valuable tool for understanding and interpreting deep neural networks used in medical image segmentation.
• The method contributes to the development of more explainable AI systems for medical image analysis.
• The approach facilitates a deeper comprehension of how deep features contribute to accurate segmentation outcomes.