Related Experiment Video
Updated: Jun 20, 2026
11:35
Dual-mode Imaging of Cutaneous Tissue Oxygenation and Vascular Function
Published on: December 8, 2010
Deep Multi-Modal Skin-Imaging-Based Information-Switching Network for Skin Lesion Recognition.
Yingzhe Yu1, Huiqiong Jia2,3, Li Zhang4
1The First Affiliated Hospital of Ningbo University, Ningbo 315211, China.
Bioengineering (Basel, Switzerland)
|March 28, 2025
Summary
A new deep learning model, Multi-Modal Skin-Imaging-based Information-Switching Network (MDSIS-Net), significantly improves skin lesion diagnosis by integrating multi-modal image data. This advanced approach enhances accuracy and interpretability for better clinical decision-making in dermatology.
Area of Science:
- Artificial Intelligence in Dermatology
- Medical Image Analysis
- Deep Learning for Skin Lesion Recognition
Background:
- Rising prevalence of skin lesions necessitates improved diagnostic tools.
- Existing multi-modal algorithms for skin lesion detection have limitations in capturing dynamic interactions across features.
- Early and precise diagnosis is crucial for effective skin lesion treatment.
Purpose of the Study:
- To propose a novel deep learning framework, Multi-Modal Skin-Imaging-based Information-Switching Network (MDSIS-Net), for enhanced skin lesion recognition.
- To address the limitations of current algorithms by effectively integrating information across multiple imaging modalities.
- To improve the accuracy and interpretability of skin lesion diagnosis.
Main Methods:
- Developed MDSIS-Net, a deep learning framework utilizing a multi-scale fully shared convolutional neural network for intra-modality feature extraction.
- Incorporated an innovative information-switching module with a cross-attention mechanism to dynamically calibrate and integrate features across modalities.
- Validated the model on clinical disfiguring dermatosis data (VISIA: 5 modalities) and the Derm7pt melanoma dataset (clinical and dermoscopic images).
Main Results:
- MDSIS-Net achieved superior performance over existing methods on disfiguring dermatosis data, with an mAP of 0.967 and accuracy of 0.960.
- On the Derm7pt melanoma dataset, MDSIS-Net outperformed benchmarks with an mAP of 0.877 and accuracy of 0.907.
- Grad-CAM heatmaps demonstrated the model's interpretability, aligning with clinical diagnostic focus areas.
Conclusions:
- The proposed deep multi-modal information-switching model significantly enhances skin lesion identification by capturing cross-modal relationship features and fine-grained details.
- MDSIS-Net improves diagnostic accuracy and interpretability, advancing clinical decision-making in dermatology.
- This work provides a foundation for future developments in skin lesion diagnosis and treatment.
