Toward Foundation Models in Radiology? Quantitative Assessment of GPT-4V's Multimodal and Multianatomic Region...

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Radiology

Background:

• Large language models (LLMs) show promise in processing medical text.
• GPT-4V, a vision-language model, has potential in medical imaging analysis.
• Quantitative assessment of GPT-4V for radiologic image interpretation is needed.

Purpose of the Study:

• To quantitatively evaluate GPT-4V's performance in interpreting unseen radiologic images.
• To assess the accuracy, sensitivity, and specificity of GPT-4V's diagnostic reports.
• To compare GPT-4V's performance against human radiologists.

Main Methods:

• Retrospective analysis of single abnormal and healthy control images across neuroradiology, cardiothoracic, and musculoskeletal radiology.
• GPT-4V interpretation via API for free-text reports and binary classification tasks.
• Performance metrics (accuracy, sensitivity, specificity) compared to a first-year resident and four board-certified radiologists.

Main Results:

• GPT-4V achieved 100% accuracy in identifying imaging modality and 99.2% in anatomic region.
• Diagnostic accuracy for free-text reports varied widely (0% for pneumothorax to 90% for brain tumor).
• GPT-4V demonstrated high false-positive rates (86.5% for free-text, 67.7% for binary classification) and lower sensitivity/specificity compared to human readers.

Conclusions:

• The initial version of GPT-4V can recognize medical image content, modality, and anatomy.
• GPT-4V currently fails to reliably detect, classify, or rule out abnormalities in radiologic images.
• Further development is required for GPT-4V to be a reliable tool in diagnostic radiology.