Concordance in Basal Cell Carcinoma Diagnosis. Building a Proper Standard Reference to Train Artificial Intelligence...

Area of Science:

• Dermatology
• Artificial Intelligence
• Medical Imaging

Background:

• Reliable labels are crucial for training Artificial Intelligence (AI) tools in medicine.
• Diagnosis of basal cell carcinoma (BCC) relies on subjective dermoscopic criteria, necessitating explainable AI outputs.
• AI tools for BCC diagnosis should identify and present these dermoscopic criteria.

Purpose of the Study:

• To analyze dermatologist agreement on dermoscopic criteria for BCC diagnosis.
• To compare the performance of an AI model trained with single-expert labels versus consensus-based labels.
• To evaluate the impact of consensus labeling on AI interpretability and bias mitigation.

Main Methods:

• 1230 teledermatology images of BCC were labeled by four dermatologists.
• A consensus standard (SR) was created using Expectation Maximization from the four diagnoses.
• AI models were trained using single-expert and consensus labels, and their performance was compared on 204 new images using McNemar's test and Hamming distance.

Main Results:

• High dermatologist agreement was observed for BCC versus non-BCC classification (Kappa = 0.9079).
• Agreement was lower for specific dermoscopic criteria.
• AI models trained with consensus labels showed statistically different performance compared to those trained with individual labels.

Conclusions:

• Establishing a standard reference (SR) from multiple expert opinions minimizes individual bias in AI training data.
• Consensus-based labeling significantly enhances the interpretability of AI diagnostic tools.
• Improved AI interpretability is essential for successful clinical adoption in dermatology.