Labeling confidence for uncertainty-aware histology image classification.

Area of Science:

• Digital pathology
• Machine learning in medicine
• Computational pathology

Background:

• Deep learning models in digital pathology require large, high-quality annotated datasets.
• Expert pathologist annotation is often infeasible, necessitating the use of non-expert annotators and leading to noisy labels.
• Medical imaging datasets exhibit instance-dependent noise and high inter/intra-observer variability, complicating model training.

Purpose of the Study:

• To design an uncertainty-driven labeling strategy for generating soft labels from non-expert annotators for skin cancer classification.
• To propose an uncertainty estimation-based framework to effectively handle noisy labels in medical imaging.
• To introduce a novel dual-branch min-max entropy calibration to penalize inexact labels during training.

Main Methods:

• Generated soft labels from 10 non-expert annotators for multi-class skin cancer classification using an uncertainty-driven strategy.
• Developed an uncertainty estimation-based framework to manage noisy labels derived from soft annotations.
• Implemented a dual-branch min-max entropy calibration to penalize inexact labels during model training.

Main Results:

• Utilizing soft labels with standard cross-entropy loss improved F1-score by approximately 4.0%.
• Calibrating the model with the proposed min-max entropy method further increased the F1-score by approximately 6.6%.
• The proposed strategy achieved significant performance improvements with negligible annotation and computational costs.

Conclusions:

• The uncertainty-driven labeling strategy and proposed framework effectively handle noisy labels in digital pathology.
• Soft labels and min-max entropy calibration offer a cost-effective solution for improving deep learning model performance in medical applications.
• This approach demonstrates a promising pathway for leveraging non-expert annotations in large-scale medical image analysis.