Training ChatGPT for surgical decisions: Bariatric surgery analysis using algorithms and evidence

Area of Science:

• Bariatric Surgery
• Artificial Intelligence in Medicine
• Surgical Decision Support Systems

Background:

• Bariatric surgery (BS) selection is complex and individualized.
• Artificial intelligence (AI) tools like ChatGPT may offer decision support, but clinical utility is unproven.
• Evaluating AI's role in optimizing BS technique selection is crucial.

Purpose of the Study:

• To assess if ChatGPT's bariatric surgery recommendations improve after AI training on scientific literature.
• To compare ChatGPT's recommendations with actual clinical decisions in bariatric surgery.
• To determine the clinical utility of AI in bariatric surgery decision-making.

Main Methods:

• Retrospective analysis of 283 primary bariatric surgery patients (2023-2025).
• ChatGPT recommended BS techniques before and after training on 412 scientific articles.
• Recommendations were compared to actual clinical decisions using percentage agreement and Cohen's Kappa.

Main Results:

• Initial ChatGPT recommendations favored sleeve gastrectomy (SG), with 20% concordance to clinical practice.
• After training, recommendations shifted, with SG decreasing and SADI-S emerging.
• Concordance improved to 25.8%, but remained low, indicating limited alignment with clinical judgment.

Conclusions:

• ChatGPT's bariatric surgery recommendations adapt post-training but show low concordance with clinical decisions.
• AI shows potential as an educational tool for bariatric surgery.
• Current AI models are not reliable for autonomous surgical decision-making in bariatric surgery.