KG -augmented executable CoT for mathematical coding.

Area of Science:

• Artificial Intelligence
• Computer Science
• Computational Linguistics

Background:

• Large language models (LLMs) demonstrate proficiency in natural language processing.
• LLMs face significant challenges in complex reasoning tasks like mathematical reasoning and code generation.
• Existing methods often lack interpretability and verifiability.

Purpose of the Study:

• To introduce a neuro-symbolic paradigm for enhanced mathematical reasoning in LLMs.
• To propose KGA-ECoT (Knowledge Graph Augmented Executable Chain-of-Thought) to address LLM limitations.
• To improve the interpretability and verifiability of LLM reasoning processes.

Main Methods:

• Decomposing mathematical problems into a structured task graph.
• Implementing an adaptive, on-demand GraphRAG for precise knowledge retrieval from symbolic reasoning libraries.
• Generating verifiable code for computational accuracy and external execution.

Main Results:

• KGA-ECoT significantly outperforms existing prompting methods on mathematical reasoning benchmarks.
• Achieved absolute accuracy improvements ranging from several to over ten percentage points.
• Demonstrated the critical roles of GraphRAG and external code execution in enhancing performance.

Conclusions:

• The neuro-symbolic paradigm, as implemented in KGA-ECoT, is effective for complex reasoning tasks.
• KGA-ECoT enhances LLM performance, interpretability, and verifiability.
• The proposed approach offers a promising direction for advancing AI reasoning capabilities.