Graph-of-Agents: A Graph-based Framework for Multi-Agent LLM Collaboration

Area of Science:

• Artificial Intelligence
• Computer Science
• Natural Language Processing

Background:

• The increasing number of large language models (LLMs) necessitates effective orchestration for improved task performance.
• Existing frameworks like Mixture-of-Agents (MoA) face challenges in agent selection, communication, and response integration.

Purpose of the Study:

• To introduce Graph-of-Agents (GoA), a novel graph-based framework for multi-agent LLM communication.
• To address the limitations of current multi-agent LLM coordination strategies.

Main Methods:

• GoA employs node sampling using model cards to select relevant agents.
• It constructs agent communication pathways by evaluating response relevance and employs directed and reverse message passing.
• Responses are aggregated using graph-based pooling for a unified output.

Main Results:

• GoA demonstrated superior performance on diverse benchmarks (MMLU, MMLU-Pro, GPQA, MATH, HumanEval, MedMCQA) using only 3 selected agents.
• Outperformed baselines that utilized all 6 agents simultaneously.
• Achieved superior performance with a reduced number of agents compared to existing methods.

Conclusions:

• GoA offers a scalable and effective approach to multi-agent LLM communication through structured message passing.
• The graph-based framework provides an efficient method for navigating the complexities of numerous LLMs.
• GoA presents a promising solution for optimizing LLM task performance.