Related Experiment Video
Updated: Jun 5, 2026
05:16
Sleeve Gastrectomy in Mice using Surgical Clips
Published on: November 14, 2020
Large Language Models and Metabolic Bariatric Surgery: A Pilot Concordance Study Between ChatGPT and
Aminah Ahmed1, Shivam Bhanderi2,3, Robyn Westerman4
1Core Surgical Trainee, West Midlands Deanery, Birmingham, UK.
Obesity Surgery
|June 4, 2026
Summary
Large language models (LLMs) showed moderate agreement with bariatric multidisciplinary teams (MDTs) in recommending metabolic surgery operations. While LLMs can assist, human factors limit their autonomous use in clinical decision-making.
Area of Science:
- Medical Informatics
- Artificial Intelligence in Surgery
- Clinical Decision Support Systems
Background:
- Metabolic surgery operation selection is complex, involving multidisciplinary teams (MDTs) assessing multiple patient factors.
- The utility of large language models (LLMs) as decision support tools in this context is not well-established.
Purpose of the Study:
- To evaluate the performance of ChatGPT-4 Auto in replicating bariatric multidisciplinary team (MDT) decision-making for metabolic surgery.
- To compare LLM recommendations with formal MDT decisions and actual surgical outcomes.
Main Methods:
- A retrospective pilot study analyzed 100 UK NHS bariatric patient cases.
- ChatGPT-4 Auto received patient data and was prompted to recommend operations as a simulated MDT.
- Concordance was assessed against actual MDT decisions and performed surgeries using statistical tests.
Main Results:
- ChatGPT-4 Auto achieved 70% crude concordance with MDT recommendations.
- Concordance with the performed operation was 63% for ChatGPT-4 Auto and 83% for the MDT.
- Agreement beyond chance (Cohen's kappa) between ChatGPT-4 Auto and MDT was low, indicating limited statistical agreement.
Conclusions:
- ChatGPT demonstrated moderate crude agreement with bariatric MDT decision-making.
- Limited agreement beyond chance suggests LLMs are not suitable for autonomous use currently.
- LLMs show potential as adjunctive tools, requiring further large-scale evaluation for clinical reasoning in metabolic surgery.
