Quantifying uncertainty in machine learning classifiers for medical imaging.

Area of Science:

• Medical Imaging
• Machine Learning
• Computer-Aided Diagnosis

Background:

• Machine learning (ML) models are valuable in medical imaging (MI) for computer-aided diagnosis.
• Assessing the confidence or uncertainty of ML models in MI is crucial for clinical implications but often overlooked.
• This research focuses on quantifying uncertainty in convolutional neural networks (CNNs) within MI applications.

Purpose of the Study:

• To apply, validate, and explore a technique for assessing uncertainty in CNNs for medical imaging.
• To investigate the impact of class imbalance and image characteristics on uncertainty estimation.
• To correlate uncertainty metrics with established performance metrics and expert assessments.

Main Methods:

• Utilized two public imaging datasets: chest X-rays (pneumonia detection) and skin cancer images (malignant vs. benign classification).
• Explored an uncertainty measure based on experiments with varying class imbalance, sample sizes, and images near the classification boundary.
• Validated findings by examining correlations with performance metrics (accuracy, sensitivity) and comparing CNN predictions with an expert radiologist.

Main Results:

• Uncertainty was minimized when training sets had balanced classes, being ~17% lower than with imbalanced sets.
• Images closer to the classification boundary exhibited 10-15 times higher uncertainty than clearer images.
• Preliminary agreement was observed between CNN predictions and an expert radiologist on a small subset of images.

Conclusions:

• Reporting uncertainty alongside predictions is vital for ML models in medical imaging.
• The proposed uncertainty metric shows significant potential for automatically assessing classifier reliability in MI.
• This approach can enhance the trustworthiness and clinical utility of AI-driven diagnostic tools.