Visual Interpretation of Convolutional Neural Network Predictions in Classifying Medical Image Modalities

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Deep learning (DL) models are crucial for medical image analysis but lack interpretability.
• Explaining DL model behavior is essential for real-world adoption in clinical settings.

Purpose of the Study:

• To introduce Class-selective Relevance Mapping (CRM) for visualizing discriminative regions in medical images.
• To enhance the explainability of convolutional neural network (CNN) predictions.
• To improve automatic medical image modality classification for information retrieval.

Main Methods:

• Developed CRM, a novel method based on the linear sum of incremental mean squared errors (MSE) at the CNN output layer.
• CRM quantifies positive and negative contributions of spatial elements in feature maps.
• Evaluated CRM on a multi-modality CNN classifying seven image types.

Main Results:

• CRM effectively localizes and visualizes discriminative regions of interest (ROIs).
• The proposed method outperforms existing class-activation techniques in detecting and localizing ROIs.
• Class-specific ROI maps generated by CRM reveal distinct visual explanations for different image modalities.

Conclusions:

• CRM offers superior interpretability for DL models in medical imaging.
• The method facilitates automatic labeling of medical imaging modalities for enhanced information retrieval.
• CRM contributes to the reliable deployment of DL in clinical computer-aided detection and diagnosis.