Comparative Performance of GPT-5 and Gemini Models in Decision Support for Bariatric Surgery: A Simulation-Based...

Artificial intelligence (AI) models are increasingly proposed for surgical decision support, yet their reliability in high-risk domains such as bariatric surgery remains uncertain. Prior bariatric AI studies have mainly evaluated patient education, isolated question-answering, or predictive modeling rather than paired safety-focused decision simulation across the perioperative pathway.

This simulation-based paired-comparison study evaluated GPT-5 and Gemini using 60 expert-validated bariatric surgery vignettes spanning preoperative, intraoperative, and postoperative decision-making. Both models were queried under standardized low-temperature conditions. Three bariatric surgeons independently scored outputs for accuracy, safety, guideline adherence, and reasoning quality. Reporting was aligned with STROBE, with AI diagnostic-evaluation elements informed by STARD-AI and analytic transparency principles informed by TRIPOD+AI.

GPT-5 outperformed Gemini across all domains. GPT-5 achieved higher accuracy (87.3% vs. 72.4%), stronger guideline adherence (78% vs. 46%), and fewer unsafe recommendations (2.1% vs. 8.4%). Composite scores were significantly higher for GPT-5 (6.92±0.61) than for Gemini (5.43±0.88; P<0.001). Gemini was 4 times more likely to produce an unsafe action than GPT-5 (risk ratio, 4.0; 95% CI, 2.2-7.3). The largest accuracy gap occurred in intraoperative scenarios. Expert concordance was substantial for GPT-5 (κ=0.71) and moderate for Gemini (κ=0.54).

GPT-5 demonstrated superior accuracy, guideline alignment, and safety compared with Gemini in bariatric surgery decision simulations. Although these findings support the potential of advanced LLMs as supervised decision-support tools, unsafe outputs persisted, reinforcing the need for model version transparency, prospective validation, and clinician oversight before clinical deployment.