Clinical Application of Vision Transformers for Melanoma Classification: A Multi-Dataset Evaluation Study

Area of Science:

• Artificial Intelligence
• Dermatology
• Medical Imaging

Background:

• Melanoma diagnosis is challenging due to visual similarity with benign nevi.
• Current deep learning models (CNNs) have limitations in generalization due to fixed input sizes and focus on local features.
• Vision Transformers (ViT) offer potential by capturing global image relationships via self-attention.

Purpose of the Study:

• To evaluate the efficacy of a Vision Transformer (ViT-L/16) for melanoma classification.
• To assess the impact of Generative Adversarial Network (GAN) based data augmentation on ViT performance.
• To compare ViT performance against Convolutional Neural Network (CNN) baselines and a commercial system.

Main Methods:

• Fine-tuning a ViT-L/16 model on the ISIC 2019 dataset.
• Generating synthetic melanoma and nevus images using StyleGAN2-ADA for dataset expansion and class balancing.
• Evaluating model performance on an external biopsy-confirmed dataset (MN187) using ROC-AUC and DeLong's test.

Main Results:

• The baseline ViT-L/16 model achieved an ROC-AUC of 0.902 on the MN187 dataset, outperforming CNNs and a commercial system.
• Adding 46,000 GAN-generated images significantly improved the ROC-AUC to 0.915, outperforming the commercial MoleAnalyzer Pro system (p=0.032).

Conclusions:

• Vision Transformers demonstrate strong potential for melanoma classification.
• GAN-based augmentation enhances ViT performance by improving global feature representation and expanding datasets.
• ViT models combined with GAN augmentation can support the development of reliable AI clinical decision-support systems for melanoma detection.