Interpretability-Based Multimodal Convolutional Neural Networks for Skin Lesion Diagnosis

Area of Science:

• Dermatology and Medical Imaging
• Artificial Intelligence in Healthcare
• Computational Pathology

Background:

• Accurate skin lesion diagnosis is crucial for effective skin cancer screening.
• Current deep learning models for skin lesion diagnosis face challenges in generalization and interpretability.
• Existing computer-aided diagnostic tools often function as 'black boxes', limiting clinical trust and adoption.

Purpose of the Study:

• To develop an interpretable multimodal convolutional neural network (IM-CNN) for enhanced skin lesion diagnosis.
• To improve the accuracy, generalization, and interpretability of AI models in skin cancer screening.
• To facilitate man-machine collaboration in medical decision-making by providing visual explanations.

Main Methods:

• Proposed an IM-CNN model integrating patient metadata and segmented skin lesion images with domain knowledge.
• Incorporated interpretable visual modules for explaining both image and metadata inputs.
• Introduced AUC_SEN_80 as a performance metric alongside AUC, sensitivity, and specificity.
• Conducted extensive experiments on the HAM10000 dataset.

Main Results:

• The IM-CNN demonstrated superior performance compared to popular deep learning models like DenseNet and ResNet for melanoma diagnosis.
• The multimodal approach achieved an average improvement of 72% in sensitivity and 21% in AUC_SEN_80 over single-modal models.
• Visual explanations provided by the model enhanced trust and facilitated understanding for dermatologists.

Conclusions:

• The proposed IM-CNN offers significant advantages in accuracy and interpretability for skin lesion diagnosis.
• Multimodal data integration and interpretable AI are key to overcoming limitations of current deep learning models in clinical settings.
• The IM-CNN shows promise for supporting dermatologists in medical decision-making and advancing skin cancer screening.