Performance changes due to differences in training data for cerebral aneurysm detection in head MR angiography images

Area of Science:

• Medical imaging analysis
• Artificial intelligence in healthcare
• Neurology

Background:

• Computer-aided detection (CAD) software performance relies heavily on training dataset quality and quantity.
• Discrepancies between development and real-world data characteristics can significantly degrade CAD software performance.
• Investigating the impact of data variations on CAD software for cerebral aneurysm detection is crucial.

Purpose of the Study:

• To investigate how differences in training data affect the detection performance of cerebral aneurysm detection software.
• To compare the impact of single-site versus multisite training data on CAD software performance.
• To evaluate the stability and reliability of CAD software trained on diverse datasets.

Main Methods:

• Utilized three types of CAD software for cerebral aneurysm detection: 3D local intensity structure analysis, graph-based features, and convolutional neural network.
• Compared three training patterns: two single-site data trainings and one multisite data training for each CAD software.
• Conducted both internal and external evaluations to assess software performance.

Main Results:

• Training with single-site data resulted in significant and unpredictable performance fluctuations when the training dataset was altered.
• CAD software trained using multisite data consistently avoided the lowest performance levels across all tested software and datasets.
• Multisite training demonstrated greater robustness compared to single-site training.

Conclusions:

• Training cerebral aneurysm detection software using data from multiple sites is recommended for ensuring stable and reliable performance.
• Multisite data training mitigates performance degradation caused by variations in dataset characteristics.
• Diversifying training data is essential for robust medical AI applications.