Reliability of artificial intelligence in hepato-pancreato-biliary clinical decision-making: a retrospective...

The integration of Artificial Intelligence (AI) into clinical decision-making is accelerating, yet its reliability compared to Multidisciplinary Team (MDT) meetings in Hepato-Pancreato-Biliary (HPB) surgery remains underexplored. This study evaluated the concordance between AI recommendations and MDT consensus in real-world HPB oncology.

A retrospective comparative analysis of all HPB cases discussed at the MDT meetings of the 1st Department of Surgery, National and Kapodistrian University of Athens, between July 1st and December 31st 2025, was conducted. Anonymized text-only clinical summaries were processed using a large language model (Gemini 3.1 Pro) to generate treatment recommendations. The primary endpoint was the agreement rate between the AI and the MDT decision.

An overall agreement rate of 91.7% (k = 0.841; p < 0.01) was observed. Concordance was exceptionally high in straightforward cases (96.7%; k = 0.938; p < 0.01), mirroring standard oncologic workflows. Conversely, in complex scenarios agreement dropped significantly to 62.5% (k = 0.304; p = 0.09). In these instances, AI recommendations diverged from the individualized, clinical judgment applied by the MDT.

AI platforms demonstrate high concordance with MDTs in routine HPB scenarios, suggesting potential for streamlining standard workloads. Nevertheless, human multidisciplinary expertise remains indispensable for complex cases requiring nuanced clinical intuition.