Standalone AI Versus AI-Assisted Radiologists in Emergency ICH Detection: A Prospective, Multicenter Diagnostic...

Area of Science:

• Medical imaging
• Artificial intelligence in healthcare
• Neurology

Background:

• Intracranial hemorrhages (ICHs) demand rapid diagnosis for optimal patient outcomes.
• Artificial intelligence (AI) offers potential solutions for neuroimaging challenges, including radiologist shortages and increased workloads.
• This study directly compares the diagnostic efficacy of standalone AI versus AI-assisted radiologists in ICH detection.

Purpose of the Study:

• To evaluate and compare the diagnostic performance of three commercial AI services against AI-assisted radiologists in identifying intracranial hemorrhages (ICHs) on brain CT scans.
• To assess the effectiveness of AI as a primary diagnostic tool versus an auxiliary aid in neuroimaging.

Main Methods:

• A prospective, multicenter study involving 3409 brain CT scans from 67 medical institutions over 15 months.
• Comparison of three registered AI services against radiologist interpretations aided by AI.
• Statistical analysis using McNemar's test and Cohen's h effect size to evaluate diagnostic metrics.

Main Results:

• AI-assisted radiologists demonstrated statistically superior performance across all metrics: sensitivity (98.91% vs. 95.91%), specificity (99.83% vs. 87.35%), and accuracy (99.53% vs. 90.11%).
• Radiologists with AI assistance had a 323-fold higher diagnostic odds ratio and a 73-fold lower false-positive rate (4 vs. 293 cases) compared to standalone AI.
• Complete error complementarity was observed, with AI identifying radiologist misses and vice versa, indicating synergistic potential.

Conclusions:

• Radiologists remain the gold standard for ICH diagnosis, significantly outperforming standalone AI systems.
• AI shows promise as a "second reader" to improve diagnostic sensitivity, but requires substantial refinement to reduce its high false-positive rate.
• Optimal implementation involves integrating AI as an auxiliary tool within radiologist workflows, not as an autonomous diagnostic system.