Related Experiment Videos
From heatmaps to language: Explainable skin cancer diagnosis using CNN-VLM integration with an expert-annotated
Faysal Ahmmed1, Asef Rahman Antik1, Samanta Mehnaj1
1Department of Computer Science, American International University-Bangladesh (AIUB), Dhaka 1229, Bangladesh.
Artificial Intelligence in Medicine
|August 3, 2026
Summary
This study introduces an explainable AI framework for skin cancer diagnosis. It enhances deep learning interpretability by combining lesion classification with clinically meaningful explanations, improving diagnostic trust.
Area of Science:
- Dermatology
- Artificial Intelligence
- Medical Imaging
Background:
- Current deep learning models for skin cancer diagnosis lack clinical interpretability, functioning as black boxes.
- Visual interpretation of lesion morphology is crucial for accurate skin cancer diagnosis.
Purpose of the Study:
- To develop an explainable diagnostic framework for skin cancer using AI.
- To enhance the interpretability of AI models in dermatological diagnosis.
Main Methods:
- Integrated a fine-tuned Xception network with CBAM for classification and a domain-adapted VLM for explanations.
- Utilized Grad-CAM++ for highlighting discriminative lesion regions and created an image-text corpus for VLM fine-tuning with LoRA.
- Trained the classifier on a preprocessed and augmented HAM10000 dataset.
Main Results:
- The classifier achieved 85.66% test accuracy and an AUC of 0.9392 across seven lesion classes.
- The fine-tuned VLM significantly improved explanation quality scores by 25.72 points (from 62.14 to 87.86).
Conclusions:
- The developed framework offers clinicians classification predictions, visual attention maps, and expert-aligned textual rationales.
- This represents a practical advancement toward trustworthy and explainable AI in dermatological diagnosis.
