Deep Learning Multi-Modal Melanoma Detection: Algorithm Development and Validation.

Area of Science:

• Dermatology
• Artificial Intelligence
• Medical Imaging

Background:

• Melanoma and seborrheic keratosis share visual similarities, complicating self-diagnosis for patients, particularly older individuals with disabilities.
• Delayed diagnosis due to visual confusion contributes to melanoma metastasis.

Purpose of the Study:

• To develop a novel multimodal deep learning technique for differentiating melanoma from seborrheic keratosis.
• To improve diagnostic accuracy and reduce delays in seeking medical attention for potentially cancerous lesions.

Main Methods:

• Trained and evaluated multiple deep learning models (ResNet50, InceptionV3, VGG16, custom model) using patient image data via transfer learning.
• Developed a separate deep learning model utilizing patient metadata.
• Combined image and metadata models using nonlinear least squares regression for optimal weighting and prediction.

Main Results:

• Achieved 88% accuracy on the HAM10000 dataset with the combined model.
• Metadata integration significantly reduced false-negative and false-positive rates.
• Activation map visualization confirmed model reliability by comparing diagnostic patterns with dermatologists' assessments.

Conclusions:

• The multimodal deep learning approach shows promise for accurate melanoma and seborrheic keratosis differentiation.
• Integration of patient metadata is crucial for enhancing diagnostic performance.
• Future applications could include a mobile app to facilitate early melanoma detection and reduce diagnostic delays.