An Attention-Enhanced Multimodal Hybrid Model for Skin Cancer Diagnosis Using Imaging and Clinical Data

Area of Science:

• Dermatology and Artificial Intelligence
• Medical Imaging Analysis
• Computational Pathology

Background:

• Skin cancer is a prevalent and potentially fatal disease, with metastasis posing a significant challenge.
• Early diagnosis is crucial for improving patient outcomes and reducing mortality rates.
• Existing diagnostic methods can be enhanced by advanced computational approaches.

Purpose of the Study:

• To develop and evaluate a hybrid AI model for the early and accurate diagnosis of skin cancer.
• To integrate features from both medical images and clinical data for a comprehensive diagnostic approach.
• To improve the performance of skin cancer detection using advanced machine learning techniques.

Main Methods:

• A hybrid model was developed, integrating Vision Transformer (ViT) and Convolutional Neural Network (CNN) for image feature extraction.
• Clinical data features were extracted using FT-Transformer, Excel Former, SAINT, GRANDE, PTaRL, and TabTransformer architectures.
• The model incorporated 13 diverse classifiers, fine-tuning, and channel attention mechanisms for enhanced performance on the PAD-UFES-20 dataset.
• Class weighting was applied to address class imbalance within the dataset.

Main Results:

• The developed hybrid model achieved a competitive Area Under the Curve (AUC) of 96.41%.
• Performance was validated against six CNN and four ViT models, demonstrating superior or comparable results.
• The model effectively utilized both clinical and imaging data for diagnosis.

Conclusions:

• The proposed hybrid AI model shows significant promise in assisting dermatologists with early skin cancer diagnosis.
• The integration of diverse AI architectures and feature engineering enhances diagnostic accuracy.
• This approach represents a step forward in leveraging AI for dermatological applications.