Related Experiment Video
Updated: May 10, 2026
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Agentic GPT-5.0 system outperforms standard large language models and human experts in critical care clinical
1Department of Emergency Medicine, Etimesgut şehit Sait Ertürk State Hospital, Ankara, Turkey.
Background:
Agentic artificial intelligence (AI) systems employing multi-model architectures with iterative reasoning may surpass standard single-model large language models (LLMs) in complex clinical decision-making. Comprehensive comparisons of agentic versus standard LLM deployment against human specialists in critical care remain limited.
Objective:
This simulation study compared the performance of an agentic system combining GPT-5.0 and Gemini 2.0 Flash against two standard LLMs (Gemini 2.0 Flash and GPT-4o) and human specialists in acid-base disorder interpretation and sepsis management using text-based clinical vignettes.
Methods:
Forty-five clinical vignettes (20 acid-base, 25 sepsis) developed by an independent expert panel were evaluated by: (1) Gemini 2.0 Flash (standard single-turn); (2) GPT-4o (standard single-turn); (3) an agentic system combining GPT-5.0 and Gemini 2.0 Flash with multi-step reasoning and cross-verification; and (4) 20 board-certified physicians. Responses were anonymized and assessed by two blinded graders against pre-established gold standards using an explicit scoring rubric.
Results:
For acid-base disorders, the agentic system achieved 91.0% overall accuracy (95% CI 85.2-96.8%), significantly outperforming GPT-4o (78.0%, P = .002), Gemini (74.5%, P < .001), and human specialists (83.0%, P = .038). SSC hour-1 bundle compliance was 96.8% for the agentic system versus 82.4% for GPT-4o, 79.2% for Gemini, and 90.4% for humans (all P < .05). ROC analysis demonstrated superior discrimination for the agentic system (AUC = 0.932) compared to humans (0.856), GPT-4o (0.814), and Gemini (0.786). Subgroup findings in complex case categories are exploratory given small case numbers.
Conclusions:
In this simulation study using text-based clinical vignettes, an agentic AI system combining GPT-5.0 and Gemini 2.0 Flash demonstrated significantly higher performance than standard LLM implementations and human medical specialists in structured tasks of acid-base interpretation and sepsis bundle compliance. These simulation-based findings suggest that agentic architectures may represent a promising direction for structured clinical decision support; prospective validation in real clinical environments with actual patient data is essential before implementation.
