Large Language Models Provide Accurate but Potentially Unsafe Answers to Multimodal Critical Care Medicine Board...

Area of Science:

• Artificial Intelligence in Medicine
• Critical Care Medicine
• Medical Education

Background:

• The integration of artificial intelligence (AI) into medical education and clinical practice is rapidly evolving.
• Large language models, such as ChatGPT-4o, are being explored for their potential to assist in complex medical tasks.
• Evaluating AI performance in high-stakes areas like critical care board review is crucial for safe implementation.

Purpose of the Study:

• To assess the accuracy and safety of ChatGPT-4o in answering multimodal critical care board review questions.
• To analyze ChatGPT-4o's capabilities in image interpretation, clinical reasoning, and potential for patient harm.
• To compare ChatGPT-4o's performance against pooled clinician responses in a simulated board examination setting.

Main Methods:

• An observational study utilized a validated item bank of 183 multiple-choice critical care board-style questions with clinical images.
• A custom ChatGPT-4o profile was employed in a simulated critical care board examination environment.
• AI-generated responses were evaluated by 14 experienced critical care clinicians for accuracy, reasoning, and potential harm.

Main Results:

• ChatGPT-4o achieved 74.9% accuracy, surpassing pooled clinician responses (71.1%, p=0.03).
• Strengths included question comprehension (87.4%), but significant weaknesses were found in image interpretation (61.7%) and reasoning (68.3%).
• The AI underperformed in critical care ultrasound (51.1%) and 33.3% of responses carried a risk of clinical harm, often linked to incorrect image interpretation or treatment advice.

Conclusions:

• ChatGPT-4o shows a marginal performance advantage over clinicians on critical care board questions but exhibits substantial limitations in multimodal interpretation.
• Deficiencies in reasoning and image analysis by ChatGPT-4o can lead to potentially harmful clinical conclusions.
• The AI's current capabilities necessitate caution for high-stakes clinical decision-making and medical education due to risks associated with multimodal data processing.