Effect of case selection on the performance of computer-aided detection schemes

Area of Science:

• Medical Imaging
• Artificial Intelligence in Medicine
• Diagnostic Systems

Background:

• Computer-aided diagnosis (CAD) schemes are crucial for medical image analysis.
• The performance of CAD systems is sensitive to the composition of training and testing datasets.
• Variability in testing databases can lead to inconsistent performance evaluations.

Purpose of the Study:

• To investigate the impact of clinical case selection on the measured performance of a CAD scheme for breast mass detection.
• To quantify the effect of database modifications on CAD system sensitivity and error rates.
• To highlight challenges in comparing different CAD schemes due to diverse testing methodologies.

Main Methods:

• A computerized scheme for automated breast mass detection from mammograms was utilized.
• The components of the testing database were deliberately modified.
• The sensitivity of the CAD scheme was measured across different database compositions at a fixed false positive rate.
• The impact of a 20% change in database cases on sensitivity was analyzed.

Main Results:

• CAD scheme sensitivity varied widely, from 26% to 100%, depending on the testing cases.
• A 20% alteration in the testing database reduced measured sensitivity by 15%-25%.
• Performance metrics are highly dependent on the specific cases used for testing.

Conclusions:

• Reliable estimation of CAD scheme accuracy is challenging due to database-dependent performance.
• Comparing different CAD schemes is problematic when testing databases differ.
• Short-term solutions include reporting case selection methods and image feature statistics alongside performance data.