Radiological Differential Diagnoses Based on Cardiovascular and Thoracic Imaging Patterns: Perspectives of Four Large...

Area of Science:

• Artificial Intelligence in Medical Imaging
• Radiology Decision Support Systems
• Natural Language Processing in Healthcare

Background:

• Differential diagnosis is crucial in radiology, yet challenging for early-career professionals.
• Large language models (LLMs) offer potential by accessing and contextualizing vast text-based information.
• This study evaluates LLMs for generating differential diagnoses from imaging patterns.

Purpose of the Study:

• To assess the utility of four LLMs (ChatGPT3.5, Google Bard, Microsoft Bing, Perplexity) in providing key differential diagnoses for cardiovascular and thoracic imaging.
• To compare the performance of these LLMs against expert radiologist consensus.

Main Methods:

• Fifteen cardiovascular (n=5) and thoracic (n=10) imaging patterns were selected.
• Each LLM generated the top 5 differential diagnoses for each pattern.
• LLM outputs were compared against a consensus of two cardiothoracic radiologists using concordance and acceptance metrics.

Main Results:

• Perplexity demonstrated the highest concordance (66.67%), followed by ChatGPT (65.33%) and Bing (62.67%). Bard had the lowest concordance (45.33%).
• Acceptance rates were highest for Perplexity (90.67%), Bing (89.33%), and ChatGPT (85.33%), with Bard showing the lowest (69.33%).
• Significant differences in both acceptance and concordance were observed among the evaluated LLMs.

Conclusions:

• LLM-generated differential diagnoses for cardiovascular and thoracic imaging exhibit high acceptance but variable concordance with expert consensus.
• Perplexity, Bing, and ChatGPT generally outperformed Bard in both metrics.
• Careful selection of the appropriate LLM is essential for clinical application and medical education.