Pre-trained multimodal large language model enhances dermatological diagnosis using SkinGPT-4.

Area of Science:

• Artificial Intelligence in Medicine
• Dermatology
• Medical Imaging Analysis

Background:

• Skin and subcutaneous diseases are a major global health burden.
• Accurate dermatological diagnosis is crucial for effective treatment.
• Existing diagnostic methods can be time-consuming and require specialized expertise.

Purpose of the Study:

• To develop an interactive dermatology diagnostic system named SkinGPT-4.
• To leverage multimodal large language models (LLMs) for enhanced diagnostic capabilities.
• To provide users with AI-powered analysis and treatment recommendations for skin conditions.

Main Methods:

• Alignment of a pre-trained vision transformer with the Llama-2-13b-chat LLM.
• Creation of an extensive dataset of 52,929 skin disease images with clinical concepts and doctor's notes.
• Implementation of a two-step training strategy for the multimodal LLM.
• Quantitative evaluation on 150 real-life cases in collaboration with board-certified dermatologists.

Main Results:

• SkinGPT-4 demonstrated potential in evaluating skin images and identifying disease characteristics.
• The system provided autonomous image evaluation, categorization of skin conditions, and in-depth analysis.
• Interactive treatment recommendations were generated by the system.

Conclusions:

• Multimodal large language models show promise for advancing dermatological diagnosis.
• SkinGPT-4 offers a novel approach to interactive and AI-assisted skin condition diagnosis.
• The system has the potential to support both patients and healthcare professionals in managing skin diseases.