Planet-wide performance of a skin disease AI algorithm validated in Korea

Area of Science:

• Dermatology
• Artificial Intelligence
• Medical Informatics

Background:

• Diverse skin conditions and low skin cancer prevalence necessitate robust diagnostic tools.
• Large-scale datasets are crucial for training and validating AI in dermatology.
• Real-world data offers unique insights into public health trends and disease distribution.

Purpose of the Study:

• To evaluate the performance of an AI model for diagnosing 70 skin diseases using hospital and real-world webapp data.
• To assess the sensitivity and specificity of AI in detecting skin cancer.
• To analyze global disease prevalence and public interest using webapp data.

Main Methods:

• Curated a large hospital dataset (National Information Society Agency [NIA] dataset; 70 diseases, 152,443 images).
• Collected real-world webapp data (1,691,032 requests).
• Employed a conservative evaluation method assessing hospital sensitivity and webapp specificity, treating malignancy predictions as false positives.

Main Results:

• Skin cancer sensitivity in Korea was 78.2% (NIA) and specificity was 88.0% (webapp).
• Top-1 and Top-3 accuracies for 70 diseases were 43.3% and 66.6% (NIA), respectively.
• Webapp data revealed variations in malignancy and benign tumor prevalence globally, with highest malignancy predictions in North America and most common benign tumors in Asia.

Conclusions:

• AI demonstrates potential for aiding global dermatologic surveillance.
• The developed AI tool can assist in identifying skin cancers and understanding disease prevalence worldwide.
• Combining hospital and real-world data provides a comprehensive approach to AI evaluation in dermatology.