Mitigating spatial hallucination in large language models for path planning via prompt engineering.

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Robotics

Background:

• Large Language Models (LLMs) are foundational for embodied intelligence.
• LLMs struggle with spatial reasoning and path planning in environments like mazes due to hallucination.
• Existing methods like Chain-of-Thought (CoT) have limitations in addressing these spatial hallucination issues.

Purpose of the Study:

• To develop an LLM-based technique, S2ERS, for optimal path planning in spatial reasoning tasks.
• To mitigate spatial hallucination issues in LLMs.
• To enhance the performance of LLMs in embodied intelligence applications.

Main Methods:

• S2ERS integrates entity and relation extraction with the Sarsa reinforcement learning algorithm.
• Spatial hallucination is addressed by extracting a graph structure from text-based maze descriptions.
• Prompt engineering includes the state-action value function Q and dynamic local Q-tables to guide planning and reduce token consumption.

Main Results:

• S2ERS significantly mitigates spatial hallucination in LLMs.
• Experimental evaluations on ChatGPT 3.5, ERNIE-Bot 4.0, and ChatGLM-6B show substantial improvements.
• Success rate improved by approximately 29%, and optimal rate by 19% compared to State-of-the-Art (SOTA) CoT methods.

Conclusions:

• S2ERS offers an effective solution for improving LLM spatial reasoning and path planning.
• The technique enhances LLM reliability in embodied intelligence tasks.
• S2ERS represents a significant advancement over existing CoT methods for spatial reasoning challenges.