Improving LLM performance on olympiad-level mathematics through cognitive decomposition.

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Mathematics

Background:

• Large Language Models (LLMs) exhibit limitations in multi-step mathematical reasoning, particularly at Olympiad levels, often due to logical errors.
• Existing architectures like Chain-of-Thought (CoT) struggle to consistently achieve high accuracy on complex mathematical proofs.

Purpose of the Study:

• To evaluate the impact of cognitive decomposition on LLM reasoning capabilities in Olympiad-level mathematics.
• To compare the performance of single-agent and multi-agent architectures under a cognitive decomposition framework.

Main Methods:

• Comparison of four architectures: Chain-of-Thought (CoT) Single-Agent, Metacognitive Single-Agent, Sequential Refinement Multi-Agent, and Council of Specialists Multi-Agent.
• Utilized the OlympiadBench dataset for rigorous evaluation.
• Conducted domain-level analysis and leave-one-out ablation studies to identify critical components.

Main Results:

• The Council of Specialists (65.45% accuracy) and Metacognitive Single-Agent (64.12%) significantly outperformed the CoT Single-Agent (55.15%).
• Cognitive decomposition, not multi-agent parallelism, was identified as the primary driver of reasoning improvements.
• Specific roles within the Council (Pattern Seeker, Constraint Analyst) proved critical for different mathematical domains (Number Theory, Geometry, Algebra).

Conclusions:

• Explicit cognitive decomposition enhances LLM mathematical reasoning.
• The Metacognitive Single-Agent architecture provides a computationally efficient alternative to multi-agent systems, delivering comparable reasoning performance.
• This architecture is recommended for its balance of robust reasoning and reduced computational overhead.