ORCH: many analyses, one merge-a deterministic multi-agent orchestrator for discrete-choice reasoning with EMA-guided...

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Machine Learning

Background:

• Multi-agent and ensemble methods enhance discrete-choice reasoning in large language models (LLMs).
• Current orchestration methods are often non-deterministic, costly, and lack reproducibility.
• There is a need for stable and efficient orchestration strategies for LLM-based reasoning.

Purpose of the Study:

• To introduce ORCH, a deterministic multi-agent orchestrator for LLMs.
• To improve accuracy and cost-performance in discrete-choice reasoning tasks.
• To provide a stable and reproducible orchestration framework.

Main Methods:

• ORCH employs a pool of heterogeneous LLM agents.
• A deterministic routing mechanism based on exponential moving average (EMA) performance tracking is utilized.
• Candidate answers from a selected subset of agents are merged through controlled aggregation.

Main Results:

• ORCH achieves consistent accuracy improvements over single-model baselines.
• It offers additional gains compared to high-cost single-model baselines.
• The deterministic pipeline enhances stability and reduces reliance on expensive models.

Conclusions:

• Deterministic EMA-guided routing is a practical and reproducible strategy for discrete-choice reasoning.
• The ORCH framework shows potential for extension to diverse tasks and agent pools.
• This approach offers a stable and efficient solution for multi-agent LLM orchestration.