Generalisation effects of predictive uncertainty estimation in deep learning for digital pathology
Milda Pocevičiūtė1,2, Gabriel Eilertsen3,4, Sofia Jarkman4,5
1Department of Science and Technology, Linköping University, Linköping, Sweden. milda.poceviciute@liu.se.
Scientific Reports
|May 18, 2022
Summary
Adding uncertainty estimates to deep learning (DL) models in digital pathology enhances diagnostic reliability. These estimates improve predictive performance and detect 70-90% of mispredictions, crucial for clinical deployment.
Area of Science:
- Digital pathology
- Artificial intelligence in medicine
- Machine learning for diagnostics
Background:
- Deep learning (DL) shows promise in digital pathology.
- Robustness is key for clinical deployment of DL diagnostic tools.
- Uncertainty estimation can enhance DL model reliability.
Purpose of the Study:
- Evaluate the value of uncertainty estimates in digital pathology DL predictions.
- Assess if uncertainty estimates boost predictive performance or detect mispredictions.
- Compare model-integrated (MC dropout, Deep ensembles) and model-agnostic (Test Time Augmentation) methods.
Main Methods:
- Compared MC dropout, Deep ensembles, and Test Time Augmentation (TTA) for uncertainty estimation.
- Evaluated four different uncertainty metrics.
- Focused experiments on domain shift scenarios (different medical center, underrepresented cancer subtype).
Main Results:
- Uncertainty estimates increase reliability by reducing sensitivity to classification threshold selection.
- Uncertainty estimates successfully detected 70-90% of model mispredictions.
- Deep ensembles performed best, followed closely by TTA.
Conclusions:
- Uncertainty estimation is valuable for robust DL applications in digital pathology.
- Methods like Deep ensembles and TTA are effective for providing reliable uncertainty estimates.
- These findings support the safe clinical integration of DL diagnostic solutions.
Related Concept Videos
00:59Propagation of Uncertainty from Random Error
1.1K
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
1.1K
01:10Propagation of Uncertainty from Systematic Error
912
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
912
01:19Survival Tree
167
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
167