ReasonWalker: Reasoning Iterative Vision-and-Language Navigation With Implicit Instructions

Area of Science:

• Artificial Intelligence
• Robotics
• Computer Vision

Background:

• Existing vision-and-language navigation (VLN) agents struggle to infer implicit user intentions and utilize past experiences in persistent environments.
• Current models lack the ability to reason over implicit instructions and adapt to dynamic environmental changes.

Purpose of the Study:

• To introduce ReasonWalker, a novel navigation model enabling reasoning-based navigation through implicit instructions over time.
• To enhance VLN agents' capability to understand and act upon subtle user cues and environmental context.

Main Methods:

• ReasonWalker constructs explicit, persistent scene maps for efficient renavigation and learning scene associations.
• A large language model (LLM) is employed to jointly process user instructions, agent observations, and scene maps, generating semantic navigation tokens.
• A hierarchical learning paradigm is proposed, first training navigation actions and then scene association for implicit instruction reasoning.

Main Results:

• Extensive experiments validate the effectiveness and superiority of the ReasonWalker model.
• The model demonstrates improved performance in navigation tasks requiring implicit instruction understanding.
• The proposed benchmark facilitates training and evaluation of advanced reasoning-based navigation.

Conclusions:

• ReasonWalker significantly advances the capabilities of vision-and-language navigation agents.
• The integration of LLMs and persistent scene mapping allows for more robust and intuitive navigation.
• The developed framework and benchmark pave the way for future research in implicit instruction-based navigation.