Reducing skin tone bias in dermatology AI via sketch-guided multimodal fusion

Area of Science:

• Dermatology
• Artificial Intelligence
• Medical Imaging

Background:

• AI systems for skin lesion diagnosis show performance disparities across different skin tones.
• Existing debiasing methods often require unavailable skin tone labels or can reduce accuracy.
• There is a need for AI systems that ensure diagnostic equity across diverse populations.

Purpose of the Study:

• To develop a dermatology AI system that reduces skin-tone-related performance disparities without using skin tone labels during training.
• To enhance fairness in AI-driven dermatological diagnosis for diverse skin populations.

Main Methods:

• A novel sketch-guided multimodal fusion framework combining RGB images and structural sketches.
• Separate encoders for each modality, integrated by a gated fusion module.
• Feature distillation loss to align color and structure representations.

Main Results:

• The proposed model achieved competitive accuracy and F1-score on Fitzpatrick17k and DDI datasets.
• Demonstrated lower mean disparity and reduced subgroup disparity across skin tone groups compared to baseline methods.
• Showed improved fairness in out-of-domain evaluations and consistent performance across skin tones on the DDI dataset.

Conclusions:

• The sketch-guided multimodal fusion framework shows promise in reducing skin-tone-related bias in dermatology AI.
• The approach preserves diagnostic utility without relying on explicit skin tone annotations.
• Further prospective validation is needed to establish clinical utility and real-world impact.