Related Experiment Video
Updated: May 11, 2026
05:56
Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application
Published on: April 14, 2023
3.3K
Fine-Tuning a Small Vision Language Model Using Synthetic Data for Explaining Bacterial Skin Disease Images
Shiwan Zhang1, Abdurrahim Yilmaz2, Gulsum Gencoglan3
1Hamlyn Centre, Department of Surgery and Cancer, Imperial College London, London SW7 2AZ, UK.
Diagnostics (Basel, Switzerland)
|February 27, 2026
Summary
Compact vision language models (VLMs) show promise for dermatology. Synthetic question-answer data improved a small VLM
Area of Science:
- Artificial Intelligence in Medicine
- Dermatology
- Medical Image Analysis
Background:
- Vision language models (VLMs) offer potential for medical image understanding.
- Large VLM scale hinders practical deployment in clinical settings.
- Adapting compact VLMs for specialized medical domains like dermatology is crucial.
Purpose of the Study:
- To investigate the effectiveness of a compact VLM for bacterial skin disease image understanding.
- To evaluate the impact of different supervision strategies on VLM performance.
- To assess the utility of synthetic question-answer (QA) data for VLM fine-tuning.
Main Methods:
- Curated a dermatology-specific dataset (PMC-derma-VQA-bacteria) from PMC-OA and BIOMEDICA.
- Generated synthetic QA pairs using Google's Gemini model.
- Fine-tuned a compact VLM (SmolVLM) using QA-only, caption-only, and combined QA+caption supervision.
Main Results:
- QA-only supervision achieved superior report-generation quality.
- The combined QA+caption strategy resulted in the highest diagnostic classification accuracy (70.20%).
- Synthetic QA supervision significantly enhanced VLM performance.
Conclusions:
- Compact VLMs can be effectively adapted for dermatology using targeted fine-tuning.
- Synthetic QA data is a viable method to improve VLM capabilities in medical image analysis.
- This approach supports enhanced diagnostic support in dermatology.
