Orchestrated multi agents sustain accuracy under clinical-scale workloads compared to a single agent

Area of Science:

• Artificial Intelligence
• Clinical Informatics
• Computational Medicine

Background:

• Large language models (LLMs) show promise for clinical applications but require evaluation under realistic workloads.
• Scalability and performance of LLM agent architectures in healthcare settings are not well-understood.
• Heterogeneous clinical tasks present unique challenges for single-agent LLM systems.

Purpose of the Study:

• To compare the performance of single-agent versus multi-agent LLM systems for clinical-scale workloads.
• To evaluate LLM accuracy, efficiency, and latency under varying batch sizes simulating clinical traffic.
• To assess the impact of task isolation on LLM performance in a multi-agent framework.

Main Methods:

• Two LLM configurations were tested: a single agent and an orchestrated multi-agent system.
• Tasks included data retrieval, extraction, and dosing calculations, with batch sizes ranging from 5 to 80.
• Performance metrics included pooled accuracy, token usage, and latency, analyzed across four LLM checkpoints.

Main Results:

• Multi-agent systems maintained high accuracy under load (90.6% at 5 tasks, 65.3% at 80 tasks), while single-agent accuracy declined sharply (73.1% to 16.6%).
• Significant performance differences emerged beyond 10 tasks (FDR-adjusted p < 0.01).
• Multi-agent execution reduced token usage up to 65-fold and limited latency growth compared to single-agent runs.

Conclusions:

• Lightweight orchestration of LLM agents is effective for clinical-scale workloads.
• Multi-agent systems offer superior accuracy, efficiency, and auditability compared to single-agent LLMs in healthcare.
• Task isolation in multi-agent designs prevents context interference and sustains performance.