D-EDL: Differential evidential deep learning for robust medical out-of-distribution detection

In computer-aided diagnosis, the extreme imbalance in disease incidence rates often results in the omission of rare conditions, leading to out-of-distribution (OOD) samples during testing. To prevent unreliable diagnostic outputs, detecting these OOD samples becomes essential for clinical jnsafety. While Evidential Deep Learning (EDL) and its variants have shown great promise in detecting outliers, their clinical application remains challenging due to the variability in medical images. We find that when encountering samples with high data uncertainty, the Kullback-Leibler divergence (KL) in EDL tends to suppress inherent ambiguity, resulting in an over-penalty effect in evidence estimation that impairs discrimination between ambiguous in-distribution cases and true outliers. Motivated by the confirmatory and differential diagnostic process in clinical practice, we propose Differential Evidential Deep Learning (D-EDL), a simple but effective method for robust OOD detection. Specifically, we treat KL as a confirmatory restriction and innovatively replace it with a Ruling Out Module (ROM) for differential restriction, which reduces over-penalty on ambiguous ID samples while maintaining OOD sensitivity. Considering extreme testing scenarios, we introduce test-time Raw evidence Inference (RI) to bypass instability in uncertainty estimation with refined evidence and further improve robustness and precision. Finally, we propose the Balanced Detection Score (BDS) to quantify the potential on clinical performance when optimally balancing misdiagnoses and missed diagnoses across varying sensitivities. Experimental results on ISIC2019, Bone Marrow Cytomorphology datasets and EDDFS dataset demonstrate that our D-EDL outperforms state-of-the-art OOD detection methods, achieving significant improvements in robustness and clinical applicability. Code for D-EDL is available at https://github.com/KellaDoe/Differential_EDL.