Layer by Layer: Assessing AI Diagnostic Accuracy With Incremental Case Information in Neuroradiology

Area of Science:

• Neuroradiology
• Artificial Intelligence

Background:

• Artificial intelligence (AI) shows promise for enhancing diagnostic accuracy and efficiency in radiology.
• Large language models (LLMs) are emerging as tools for medical image interpretation.

Purpose of the Study:

• To evaluate the diagnostic performance of Google Gemini (version 1.5 Flash) in challenging neuroradiology cases.
• To assess AI accuracy across different anatomical areas (brain, head and neck, spine) and with incremental data.

Main Methods:

• Analysis of 143 neuroradiology cases from the American Journal of Neuroradiology's "Case of the Month" series.
• Google Gemini was prompted to diagnose cases with varying levels of information (history, incremental imaging).
• Diagnostic accuracy was measured at each data stage and by specialty.

Main Results:

• Gemini's accuracy increased from 3.5% (history alone) to 45.7% (complete imaging).
• Highest accuracy was observed in spine cases (51.9%), followed by head and neck (45.5%) and brain (44.0%).
• Performance improvement over time was statistically significant (p < 0.0000000001).

Conclusions:

• Google Gemini demonstrates moderate diagnostic accuracy in neuroradiology, which improves with more comprehensive data.
• The findings highlight the need for ongoing validation and transparency of AI tools before clinical integration.
• This study underscores the growing role and necessity of rigorous evaluation for AI in neuroradiology.