Multi-Aspect Knowledge-Enhanced Dermatological Vision-Language Pretraining with Multi-Agent Data Generation

Vision-language pretraining (VLP) has emerged as a powerful paradigm in medical image analysis, enabling representation learning from large-scale image-text pairs without relying on expensive expertise annotations. However, these challenges are particularly pronounced when VLP is applied to dermatology. The inherent noise and sparsity of web-crawled data hinder effective pretraining, while multifaceted dermatological knowledge is challenging to exploit from long, unstructured descriptions for pretraining optimization. To address these challenges, we propose a dermatological VLP framework integrating a Multi-Agent data GENeration (MAGEN) system and Ontology-based Multi-Aspect Knowledge-Enhanced (O-MAKE) pretraining. First, MAGEN enhances data quality by synthesizing knowledge-enriched descriptions via a foundation model-assisted captioning and retrieval-based verification pipeline. Second, O-MAKE addresses the difficulty of learning from long, unstructured texts by decomposing them into distinct knowledge aspects. This facilitates fine-grained alignment at both global and patch levels, while explicitly modeling medical concept relationships through ontology-guided mechanisms. Comprehensive experiments demonstrate the effectiveness of each component, achieving state-of-the-art zero-shot performance on disease classification and cross-modal retrieval tasks across eight dermatological datasets.Our code and the augmented dataset Derm1M-AgentAug, comprising over 400K skin image-text pairs, are publicly available at https://github.com/XiejiLi/MAGEN-O-MAKE.