An Attention-Based Mechanism to Combine Images and Metadata in Deep Learning Models Applied to Skin Cancer...

Area of Science:

• Dermatology
• Artificial Intelligence
• Medical Imaging

Background:

• Current computer-aided skin cancer classification relies solely on lesion images.
• Demographic data, crucial for human expert diagnosis, is often underutilized in AI models.
• Deep learning models show promise but can be enhanced by incorporating patient metadata.

Purpose of the Study:

• To develop and evaluate a novel deep learning approach for skin cancer classification that integrates both image and metadata features.
• To introduce the Metadata Processing Block (MetaBlock) algorithm for enhancing feature relevance in classification pipelines.
• To compare the performance of MetaBlock against other data combination methods.

Main Methods:

• Proposed the Metadata Processing Block (MetaBlock) algorithm to process and integrate patient demographic data with image features.
• Employed deep learning models for skin cancer classification.
• Compared MetaBlock with feature concatenation and MetaNet approaches on two distinct skin lesion datasets.

Main Results:

• The proposed MetaBlock method consistently improved classification performance across all tested deep learning models.
• MetaBlock outperformed other combination approaches in 6 out of 10 tested scenarios.
• Demonstrated the value of integrating demographic metadata for more accurate skin cancer classification.

Conclusions:

• Combining image and metadata features using deep learning, particularly with the MetaBlock approach, enhances skin cancer classification accuracy.
• The MetaBlock algorithm offers a superior method for leveraging patient demographics in AI-driven dermatological diagnosis.
• Future research should explore further integration of diverse data types for improved medical AI systems.