Postoperative complication management: How do large language models measure up to human expertise?

Area of Science:

• Medical Informatics
• Artificial Intelligence in Surgery
• Surgical Patient Care

Background:

• Effective management of postoperative complications is crucial for surgical outcomes.
• Large Language Models (LLMs) offer potential support for medical professionals.
• Evaluating LLM capabilities in surgical patient care is an emerging area of research.

Purpose of the Study:

• To compare the performance of three state-of-the-art LLMs against human surgical caregivers in managing postoperative complications.
• To assess LLMs' accuracy in triage, diagnosis, and management planning for postsurgical patients.

Main Methods:

• Six realistic postoperative patient cases were used for evaluation.
• GPT-3, GPT-4, and Gemini-Advanced were queried and their responses compared to human expert assessments.
• Performance was evaluated based on medical correctness, coherence, and completeness of triage, diagnosis, and management plans.

Main Results:

• GPT-4 significantly outperformed humans in correctly identifying postoperative complications (96.7% vs. 76.3%).
• GPT-3 and GPT-4 provided comprehensive diagnostic and therapeutic management plans.
• Gemini-Advanced showed limitations, often censoring outputs and providing fewer recommendations.

Conclusions:

• LLMs, particularly GPT-4, demonstrate considerable competence in interpreting postoperative care scenarios.
• LLMs show potential to augment surgical routine care by providing accurate management recommendations.
• Further development of LLMs could enhance their utility in supporting surgical teams.