Investigating the Quality of DermaMNIST and Fitzpatrick17k Dermatological Image Datasets.

Area of Science:

• Dermatology
• Artificial Intelligence
• Medical Imaging

Background:

• Deep learning models achieve high accuracy in dermatological tasks.
• Large datasets are essential for training reliable deep neural networks.
• Data quality issues can significantly impact model performance and reliability.

Purpose of the Study:

• To meticulously analyze popular dermatological image datasets (DermaMNIST, HAM10000, Fitzpatrick17k) for data quality problems.
• To quantify the impact of identified data quality issues on benchmark results.
• To propose corrections for these datasets and ensure reproducibility.

Main Methods:

• Conducting detailed analyses of DermaMNIST, HAM10000, and Fitzpatrick17k datasets.
• Identifying common data quality issues: duplicates, data leakage, mislabeled images, and undefined test partitions.
• Measuring the effect of these issues on model performance metrics.

Main Results:

• Uncovered significant data quality issues across the analyzed dermatological datasets.
• Demonstrated how these issues negatively affect the benchmark results of deep learning models.
• Provided corrected versions of the datasets and a publicly available analysis pipeline.

Conclusions:

• Data quality is a critical factor for the success of deep learning in dermatology.
• Addressing data quality issues is essential for reliable diagnostic tools.
• Publicly sharing analysis methods promotes transparency and facilitates improvements in medical AI datasets.