Towards safe and reliable deep learning for lung nodule malignancy estimation using out-of-distribution detection

Area of Science:

• Medical Imaging and Artificial Intelligence
• Machine Learning for Healthcare
• Clinical Decision Support Systems

Background:

• Artificial Intelligence (AI) models risk performance degradation with dataset shift, where unseen data differs from training data.
• Detecting out-of-distribution (OOD) data is crucial for the safe and reliable clinical deployment of AI.
• Existing methods for OOD detection may not be sufficiently robust across diverse dataset shifts.

Purpose of the Study:

• To propose and evaluate a Mahalanobis distance (MD)-based method for OOD detection in deep learning (DL) models.
• To compare the performance of the proposed MD method against classical OOD detection techniques.
• To assess the method's effectiveness in detecting OOD data in a clinical AI application for lung nodule malignancy risk estimation.

Main Methods:

• Implemented a Mahalanobis distance (MD) approach to measure sample similarity against in-distribution feature distributions.
• Integrated the MD method into a deep learning (DL) model for chest CT lung nodule analysis.
• Validated the method across four distinct dataset shifts known to impact AI performance, comparing it with classical OOD detection methods.

Main Results:

• The proposed Mahalanobis distance (MD) method significantly outperformed classical approaches in detecting both near- and far-OOD samples.
• Effective OOD detection was achieved across all tested datasets exhibiting varied data distribution shifts.
• The MD method demonstrated seamless integration of additional in-distribution (ID) data without compromising OOD detection accuracy.

Conclusions:

• The Mahalanobis distance (MD) method offers a robust and effective solution for OOD detection in clinical AI.
• This approach enhances the reliability and safety of AI models when encountering data distribution shifts.
• The study suggests that MD-based OOD detection can maintain DL model performance even with increasing OOD scores.