ORCH: many analyses, one merge-a deterministic multi-agent orchestrator for discrete-choice reasoning with EMA-guided...

Multi-agent/ensemble approaches can improve discrete-choice reasoning with large language models, but common orchestration methods are often non-deterministic, expensive, and difficult to reproduce. We propose ORCH, a deterministic multi-agent orchestrator that targets higher accuracy and better cost-performance via stable routing.

ORCH uses a pool of heterogeneous LLM agents and a deterministic routing mechanism based on exponential moving average (EMA) performance tracking. For each question, ORCH selects a small subset of agents, obtains candidate answers, and merges them through a controlled aggregation procedure. We evaluate ORCH on multiple discrete-choice benchmarks and compare against single-model baselines and non-routed ensemble strategies under consistent prompting and scoring.

ORCH delivers consistent accuracy improvements over the best low-cost single model and provides additional gains over high-cost single-model baselines on several tasks, while reducing reliance on always-invoking expensive models. The deterministic routing and merge pipeline improves stability across runs.

ORCH demonstrates that deterministic EMA-guided routing can offer a practical and reproducible orchestration strategy for discrete-choice reasoning. This framework can be extended to additional tasks, agent pools, and preference-aware routing policies in future work.