Incorporating uncertainty in learning to defer algorithms for safe computer-aided diagnosis

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Clinical Decision Support

Background:

• Deep neural networks (DNNs) are vital for computer-aided diagnosis.
• Erroneous diagnoses from DNNs carry significant patient risks.
• Existing methods lack optimal uncertainty handling for deferral.

Purpose of the Study:

• Introduce a novel 'learning to defer with uncertainty' (LDU) algorithm.
• Enhance diagnostic network reliability by deferring uncertain cases.
• Improve patient safety in AI-driven diagnostic systems.

Main Methods:

• Developed the LDU algorithm to identify and defer high-uncertainty patients.
• Evaluated LDU on myocardial infarction, comorbidity, and chest imaging diagnoses.
• Compared LDU against 'learning to defer without uncertainty' (LD) and 'direct triage by uncertainty' (DT) methods.

Main Results:

• LDU matched LD's F1 score but significantly reduced deferral rates (e.g., 36% vs. 69% for pleural effusion).
• LDU increased F1 score by 17% in cases with high-confidence erroneous diagnoses where DT was inapplicable.
• The deferral weight in LDU is adjustable for balancing accuracy and deferral rates.

Conclusions:

• LDU effectively reduces erroneous diagnoses in clinical practice.
• The algorithm can be integrated into existing diagnostic networks.
• LDU offers a tunable approach to manage diagnostic uncertainty and improve patient outcomes.