Vision-Language Navigation Policy Learning and Adaptation.

Area of Science:

• Artificial Intelligence
• Robotics
• Computer Vision

Background:

• Vision-language navigation (VLN) enables embodied agents to follow natural language instructions in 3D environments.
• Key challenges include cross-modal grounding, handling ambiguous feedback, and generalizing to new environments.

Purpose of the Study:

• To develop novel methods for addressing critical challenges in vision-language navigation.
• To improve the agent's ability to ground language instructions to visual scenes and generalize to unseen environments.

Main Methods:

• Proposed Reinforced Cross-Modal Matching (RCM) using reinforcement learning for local and global grounding.
• Introduced Self-Supervised Imitation Learning (SIL) to enhance policy generalization in unseen environments.

Main Results:

• RCM achieved state-of-the-art performance, improving Success Rate weighted by Path Length (SPL) by 10% over baselines.
• SIL significantly reduced the performance gap between seen and unseen environments from 30.7% to 11.7%.

Conclusions:

• The proposed RCM and SIL methods effectively address core challenges in VLN.
• These advancements lead to more robust and generalizable embodied agents capable of complex navigation tasks.