Reducing skin tone bias in dermatology AI via sketch-guided multimodal fusion

AI-driven skin lesion diagnosis systems are revolutionizing dermatology practice but perform worse on darker skin populations, which threatens diagnostic equity in dermatology. Existing debiasing strategies rely on explicit skin tone annotations or adversarial removal of demographic information, which may be unavailable in practice and can harm diagnostic accuracy. We aimed to design a dermatology AI system that reduces skin-tone-related performance disparities across diverse skin populations without using skin tone labels during training. We propose a novel sketch-guided multimodal fusion framework that combines color (RGB) images with algorithmically generated structural sketches. Separate encoders extract representations from each modality, which are integrated by a gated fusion module that adaptively weighs color and structure features. A feature distillation loss aligns color features with their sketch counterparts to encourage structure-aware representations while retaining clinically relevant color cues. We trained and evaluated the model on the Fitzpatrick17k and Diverse Dermatology Images (DDI) datasets. The fairness performance was assessed with Equalized Opportunity, Equalized Odds, and Predictive Quality Disparity across skin tone groups. Out-of-domain robustness was examined using a DermaAmin and Atlas Dermatologico split. On Fitzpatrick17k, the proposed model yielded competitive accuracy and F1-score, while showing lower mean disparity in fairness evaluation than baseline methods. It also demonstrated reduced subgroup disparity across skin tone groups on the evaluated fairness metrics. In the out-of-domain evaluation setup, the model also exhibited improved fairness. On the DDI dataset, the framework showed consistent performance across different skin tone groups with reduced variation. Our proposed model shows promise in reducing skin-tone-related bias in dermatology while preserving utility without relying on explicit skin tone annotations. The observed improvements in skin tone fairness across two datasets suggest that our approach may help reduce measured subgroup disparities in automated skin lesion assessment, although clinical utility and real-world impact remain to be established through prospective validation.