Visualizing Multimodal Deep Learning for Lesion Prediction.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Neuroscience

Background:

• U-Nets, a type of convolutional neural network, excel in medical image segmentation.
• Neural networks are often 'black boxes,' hindering interpretation, especially for non-computer scientists.
• Interpreting AI models is crucial for clinical adoption and trust.

Purpose of the Study:

• To develop a visual system for interpreting U-Nets used in stroke brain lesion prediction.
• To enable users to examine U-Net computations and results on multimodal imaging data.
• To provide insights for improving training datasets and understanding learned features.

Main Methods:

• Developed a visual system with multiple views for U-Net examination.
• Enabled loading of trained U-Nets and running them on patient data.
• Integrated visualization of the U-Net's computational process.

Main Results:

• The system allows interactive exploration of U-Net predictions for stroke lesions.
• Visualizations revealed how the neural network processes multimodal imaging data.
• Insights were gained into potential improvements for the training database and feature learning.

Conclusions:

• The visual system enhances the interpretability of U-Nets in medical imaging.
• This tool facilitates collaboration between AI developers and medical professionals.
• It aids in optimizing AI models for stroke lesion detection and analysis.