The Effect of Structured Context on Chest Radiograph Interpretation by a Multimodal Large Language Model: A Pilot...

Area of Science:

• Artificial Intelligence in Radiology
• Medical Image Interpretation
• Large Language Models

Background:

• Multimodal large language models (LLMs) show promise for medical image interpretation.
• The impact of structured contextual guidance on LLM performance in radiology is not well understood.

Purpose of the Study:

• To evaluate if structured contextual guidance enhances ChatGPT's performance in interpreting chest radiographs.
• To assess the reliability of LLMs for diagnostic tasks in medical imaging.

Main Methods:

• A comparative pilot study involving 50 chest radiographs with known diagnoses.
• ChatGPT was tested in a baseline condition (standard prompt) and a structured-context condition (teaching module + radiologic framework).
• Performance was measured using percent-correct scoring with partial credit for appropriate reasoning.

Main Results:

• Overall accuracy increased from 28% to 62% with structured guidance.
• 100% accuracy was achieved for COPD, pulmonary edema, and foreign body identification.
• Performance was limited for lung cancer (20%), feeding tubes (40%), and rib fractures; atelectasis accuracy decreased.

Conclusions:

• Structured contextual guidance significantly improved ChatGPT's performance on specific chest radiograph interpretation tasks.
• Current LLM diagnostic reliability is insufficient for independent clinical use.
• Further research with larger studies, standardized scoring, and external validation is necessary.