NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning.

Area of Science:

• Artificial Intelligence
• Robotics
• Natural Language Processing

Background:

• Vision-and-Language Navigation (VLN) is a key challenge in Embodied AI, requiring agents to follow language instructions in 3D spaces.
• Large Language Models (LLMs) show promise for VLN but suffer from domain gaps when used offline.
• Existing methods often struggle with the domain gap between VLN tasks and LLM training data.

Purpose of the Study:

• To propose Navigational Chain-of-Thought (NavCoT), a novel strategy for parameter-efficient, in-domain training of LLMs for VLN.
• To mitigate the domain gap and enable self-guided navigational decision-making in embodied agents.
• To improve the accuracy and cost-effectiveness of LLM-based agents in complex 3D environments.

Main Methods:

• NavCoT prompts LLMs to forecast a navigational chain-of-thought at each timestep.
• The LLM acts as a world model to predict the next observation, selects the best matching observation, and determines actions based on reasoning.
• Formalized labels are constructed for training, enabling the LLM to generate desired chain-of-thought outputs for improved action decisions.

Main Results:

• NavCoT significantly outperforms direct action prediction variants across popular VLN benchmarks like R2R, RxR, and R4R.
• Parameter-efficient fine-tuning of NavCoT achieved approximately 7% relative improvement over a GPT-4 based approach on the R2R dataset.
• The proposed method effectively simplifies action prediction through disentangled reasoning.

Conclusions:

• NavCoT offers a cost-effective solution to bridge the domain gap for LLMs in VLN tasks.
• The approach enhances navigational reasoning and decision-making capabilities of embodied agents.
• NavCoT paves the way for more task-adaptive, scalable LLM-based embodied agents applicable to real-world robotics.