Instance-level quantitative saliency in multiple sclerosis lesion segmentation.

Area of Science:

• Medical Imaging Analysis
• Artificial Intelligence in Healthcare
• Neuroscience

Background:

• Explainable AI (XAI) methods are developing for classification and segmentation tasks.
• Existing XAI methods lack instance-level explanations for semantic segmentation, especially in medical imaging for specific lesions.
• Understanding model decisions for individual lesions is vital for multi-lesional diseases.

Purpose of the Study:

• To develop and validate instance-level explanation maps for semantic segmentation.
• To apply these methods to the segmentation of white matter lesions (WML) in multiple sclerosis (MS) using MRI data.
• To quantitatively assess the model's reliance on different MRI sequences and identify potential errors.

Main Methods:

• Extended SmoothGrad and Grad-CAM++ to create instance-level explanation maps with quantitative saliency.
• Applied methods to WML segmentation using 3D U-Net, nnU-Net, and Swin UNETR on 4023 MRI scans from 687 MS patients.
• Computed saliency maps and analyzed their distributions across different prediction types (TP, FN, FP, TN).

Main Results:

• Instance saliency maps demonstrated that models prioritize FLAIR over MPRAGE for WML segmentation.
• Saliency maps indicated that FLAIR hyperintensity and surrounding healthy white matter are key for WML detection.
• Quantitative analysis of saliency map peaks showed significant differences between true positive, false positive, false negative, and true negative predictions, suggesting error identification capabilities.

Conclusions:

• Introduced novel XAI methods for quantitative, instance-level explanations in semantic segmentation.
• The proposed XAI maps are architecture-agnostic and can improve model performance, optimize architecture, and enhance user trust.
• These methods provide lesion-specific justifications for AI decisions in medical image analysis.