Related Experiment Video
Updated: Jul 10, 2026
09:53
Quantifying the Brain Metastatic Tumor Micro-Environment using an Organ-On-A Chip 3D Model, Machine Learning, and Confocal Tomography
Published on: August 16, 2020
Comprehensive Performance Testing and External Validation of an AI Algorithm to Detect and Segment Brain Metastases.
Rupesh Kotecha1,2, Eyub Y Akdemir1, Naseem Ud Din1
1Department of Radiation Oncology, Miami Cancer Institute, Baptist Health South Florida, Miami, Florida, USA.
Neuro-Oncology
|July 9, 2026
Summary
This study validated an artificial intelligence (AI) algorithm for brain metastasis imaging, finding promising performance but recommending physician oversight due to contour revision rates. The AI tool shows potential as a human-in-the-loop system.
Area of Science:
- Radiology
- Medical Imaging
- Artificial Intelligence
Background:
- AI models show promise in brain metastasis imaging but lack external validation.
- This study addresses limitations in AI model generalizability and adoption through comprehensive testing.
Purpose of the Study:
- To perform comprehensive performance testing and external validation of a U-Net-based AI algorithm for brain metastasis detection.
- To evaluate the AI algorithm's accuracy, reliability, and clinical utility against reference standards.
Main Methods:
- A U-Net-based AI model underwent FDA clearance performance testing on a multi-institutional cohort.
- External validation utilized augmented dual-sequence imaging and an open-access dataset (UCSF-BMSR).
- Evaluation metrics included sensitivity, false positive rate, Dice Similarity Coefficient, Hausdorff distance, and physician assessment.
Main Results:
- The AI algorithm achieved 90.0% sensitivity and 0.86 DSC in FDA testing.
- External validation showed sensitivities of 81.4% and 85.2% with DSCs of 0.70 and 0.78.
- A significant rate (46.3%) of contour revisions were noted in the augmented external validation cohort.
Conclusions:
- The AI algorithm demonstrates promising performance across diverse datasets.
- Findings suggest the AI's role as a human-in-the-loop tool, requiring physician oversight rather than autonomous operation.
- Further refinement is needed to minimize contour revision rates for broader clinical integration.