Towards Deviation-Robust Agent Navigation via Perturbation-Aware Contrastive Learning.

Area of Science:

• Robotics
• Artificial Intelligence
• Computer Vision

Background:

• Vision-and-language navigation (VLN) agents follow language instructions in 3D environments.
• Current VLN agents struggle with real-world disturbances like obstacles or interruptions, leading to route deviations.
• Existing models lack robustness in dynamic, unpredictable navigation scenarios.

Purpose of the Study:

• To enhance the generalization and robustness of VLN agents in real-world navigation.
• To develop a model-agnostic training paradigm for deviation-robust navigation.
• To improve VLN agent performance under various environmental perturbations.

Main Methods:

• Introduced Progressive Perturbation-aware Contrastive Learning (PROPER), a novel training paradigm.
• Implemented a path perturbation scheme to simulate route deviations.
• Designed a progressively perturbed trajectory augmentation strategy for adaptive learning.
• Developed a perturbation-aware contrastive learning mechanism to differentiate perturbed and unperturbed trajectories.

Main Results:

• PROPER improved state-of-the-art VLN baselines in perturbation-free scenarios on the Room-to-Room (R2R) benchmark.
• A new dataset, Path-Perturbed R2R (PP-R2R), was created to evaluate robustness.
• Experiments on PP-R2R demonstrated the limitations of current VLN agents' robustness and PROPER's effectiveness in enhancing it.

Conclusions:

• PROPER significantly enhances the generalization ability of VLN agents to real-world navigation challenges.
• The proposed method improves deviation-robustness, a critical factor for practical VLN applications.
• Further research and datasets like PP-R2R are needed to address the robustness gap in VLN.