Dermatologist versus artificial intelligence confidence in dermoscopy diagnosis: Complementary information that may...

Area of Science:

• Dermatology
• Artificial Intelligence
• Machine Learning

Background:

• Deep learning models in dermatology often lack uncertainty quantification.
• Standard neural networks provide class probabilities but do not model prediction uncertainty.
• Quantifying uncertainty is crucial for reliable AI-assisted medical diagnosis.

Purpose of the Study:

• To train a stochastic neural network for skin lesion classification.
• To evaluate the diagnostic performance and uncertainty estimation of the network.
• To compare the network's performance and uncertainty to dermatologists' assessments.

Main Methods:

• Developed a novel method to quantify uncertainty in stochastic neural networks.
• Trained a stochastic neural network for skin lesion classification.
• Compared the network's diagnostic performance and uncertainty scores with those of 29 dermatologists.

Main Results:

• The neural network achieved 50% sensitivity and 88% specificity, comparable to average dermatologists (68% sensitivity, 73% specificity).
• Higher confidence/lower uncertainty correlated with better diagnostic performance in both the network and dermatologists.
• No significant correlation was found between the neural network's uncertainty and dermatologists' confidence (R = -0.06, p = 0.77).

Conclusions:

• Stochastic neural networks can provide uncertainty estimates for skin lesion classification.
• Dermatologists should exercise caution with AI outputs, especially when uncertainty is high.
• Integrating uncertainty scores into AI systems can enhance human-computer interaction in clinical settings.