Textual Proficiency and Visual Deficiency: A Comparative Study of Large Language Models and Radiologists in MRI...

Area of Science:

• Artificial Intelligence in Medical Imaging
• Radiology and Diagnostic Imaging
• Machine Learning for Healthcare

Background:

• Large Language Models (LLMs) show promise in various medical applications.
• Assessing LLM capabilities in specialized diagnostic tasks like MRI artifact detection is crucial.
• Current LLMs' performance in visual interpretation tasks remains a key area of investigation.

Purpose of the Study:

• To evaluate the performance of multiple LLMs against radiologists in detecting and correcting MRI artifacts.
• To compare LLM and radiologist capabilities using both text-based and visual question-answering formats.
• To assess the temporal consistency of LLM and radiologist responses over time.

Main Methods:

• A cross-sectional study involving six LLMs and five radiologists.
• Participants completed text-based and visual evaluations of MRI artifacts across two rounds.
• Responses were scored using Likert scales for management and correction, with statistical comparisons.

Main Results:

• LLMs significantly outperformed radiologists in text-based MRI artifact questions.
• Radiologists demonstrated superior accuracy in visually identifying and correcting MRI artifacts.
• Top LLMs achieved limited accuracy (18-20%) in visual tasks, while radiologists exceeded 90%.

Conclusions:

• LLMs are proficient in text-based artifact knowledge but lack visual diagnostic capabilities.
• Current LLMs are not suitable for independent MRI artifact diagnosis.
• LLMs show potential as educational tools or in human-in-the-loop systems, pending multimodal AI advancements.