Multiclass Classification of Pigmented Skin Lesions Using a Multimodal Large Language Model.

Area of Science:

• Artificial Intelligence in Dermatology
• Medical Image Analysis
• Computational Pathology

Background:

• Pigmented skin lesions present diagnostic challenges due to visual similarities in standard photographs.
• Multimodal large language models (MMLLMs) show potential as image-based diagnostic aids, especially in resource-limited settings.
• Dermoscopy, while effective, is not universally accessible, highlighting the need for alternative diagnostic tools.

Purpose of the Study:

• To evaluate the accuracy of a fine-tuned MMLLM in classifying eight common pigmented skin conditions using only clinical photographs.
• To assess the potential of MMLLMs as a non-dermoscopic diagnostic support tool for pigmented lesions.
• To provide a scalable solution for preliminary skin lesion assessment.

Main Methods:

• A multimodal large language model (InstructBLIP-flan-t5-xl) was fine-tuned using a curated dataset of 979 pigmented skin lesion images.
• The dataset included eight lesion types: acquired dermal melanocytosis, basal cell carcinoma, ephelis, malignant melanoma, melasma, nevus, seborrheic keratosis, and solar lentigo.
• Model performance was evaluated using macro-average sensitivity, specificity, F1 score, and ROC AUC, with lesion labels masked during training.

Main Results:

• The fine-tuned MMLLM achieved a macro-average sensitivity of 86.0%, specificity of 98.2%, and F1 score of 0.86 on the validation set.
• Area under the receiver operating characteristic curve (ROC AUC) exceeded 0.95 for six of the eight lesion classes.
• Malignant melanoma demonstrated high performance (94% sensitivity, 0.98 ROC AUC), while nevus had lower sensitivity (78%, 0.89 ROC AUC).

Conclusions:

• Fine-tuned MMLLMs can accurately classify common pigmented skin lesions using only clinical photographs.
• This AI-driven approach offers rapid diagnostic support, particularly valuable in settings lacking access to dermoscopy.
• Future research should focus on expanding dataset diversity, conducting multicenter validation, and assessing real-world clinical utility.