STAMP: Spatial-Temporal Anchored Motion Planning for Zero-Shot Continuous Vision-and-Language Navigation.

Area of Science:

• Robotics
• Artificial Intelligence
• Computer Vision

Background:

• Vision-and-Language Navigation in continuous environments (VLN-CE) demands agents ground language into motion decisions under partial observability.
• Multimodal Large Language Models (LVLMs) excel at semantic understanding but struggle with spatial grounding, embodied memory, and geometric constraints in VLN.

Purpose of the Study:

• To propose STAMP, a Spatial-Temporal Anchored Motion Planning framework, to bridge the gap between pretrained world knowledge and embodied navigation for zero-shot VLN-CE.
• To enhance LVLM performance in VLN by addressing limitations in spatial grounding and decision-making.

Main Methods:

• STAMP employs a hierarchical design decoupling semantic reasoning from motion execution, utilizing a frozen LVLM over a structured abstraction.
• A novel multimodal spatial-temporal anchoring mechanism encodes landmarks, action semantics, depth geometry, and navigation history.
• An explicit Chain-of-Navigation reasoning process and an online, backtracking-enabled topological map support robust planning.

Main Results:

• STAMP achieves performance comparable to state-of-the-art zero-shot methods on VLN-CE benchmarks.
• The framework demonstrates effectiveness in both simulated and real-world navigation tasks.
• STAMP successfully grounds natural language instructions into reliable long-horizon motion decisions.

Conclusions:

• STAMP effectively enhances zero-shot vision-and-language navigation by integrating spatial-temporal reasoning and explicit planning mechanisms.
• The proposed framework offers a systematic approach to improve embodied agent navigation in complex, continuous environments.
• STAMP provides a robust solution for VLN-CE by addressing key challenges in perception, grounding, and decision-making.