Orchestrated multi agents sustain accuracy under clinical-scale workloads compared to a single agent

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Informatics
• Natural Language Processing

Background:

• Large language models (LLMs) show promise for clinical applications.
• Evaluating LLM performance under real-world clinical workloads is crucial.
• Scalability and efficiency are key challenges for LLM deployment in healthcare.

Purpose of the Study:

• To compare the performance of single-agent versus multi-agent LLM designs for clinical tasks.
• To assess the impact of batch size on LLM accuracy and efficiency in a clinical setting.
• To determine if a multi-agent orchestration approach can maintain performance under heavy clinical loads.

Main Methods:

• Two LLM architectures were tested: a single agent and a multi-agent orchestrator.
• Tasks included data retrieval, extraction, and dosing, with batch sizes from 5 to 80.
• Performance was measured by accuracy, token usage, and latency.

Main Results:

• Multi-agent LLM accuracy remained high (90.6% at 5 tasks, 65.3% at 80 tasks).
• Single-agent LLM accuracy significantly decreased (73.1% to 16.6%, p < 0.01).
• Multi-agent systems used up to 65-fold fewer tokens and showed limited latency growth.

Conclusions:

• Lightweight orchestration in a multi-agent LLM system preserves accuracy and efficiency.
• Multi-agent LLM designs are superior to single-agent designs for mixed-task clinical workloads.
• This approach offers a scalable and efficient solution for deploying LLMs in clinical settings.