Visual Perception Generalization for Vision-and-Language Navigation via Meta-Learning.

Area of Science:

• Robotics
• Artificial Intelligence
• Computer Vision

Background:

• Vision-and-language navigation (VLN) agents navigate environments using language and real-time visual input.
• Existing VLN methods often use fixed camera configurations, limiting skill transfer to robots with different visual setups.
• Real-world robots exhibit diverse camera configurations, creating a visual information gap that hinders navigation skill generalization.

Purpose of the Study:

• To develop a visual perception generalization strategy for VLN agents.
• To enable agents to rapidly adapt to novel camera configurations using meta-learning.
• To compare the effectiveness of different meta-learning algorithms for visual perception generalization.

Main Methods:

• Proposed a meta-learning strategy to address visual perception generalization in VLN.
• Localized the generalization challenge to the visual perception module.
• Compared Model-Agnostic Meta-Learning (MAML) for few-shot adaptation and a metric-based method with an affine transformation (AT) layer.

Main Results:

• The proposed strategy successfully adapted learned navigation skills to new camera configurations.
• Experimental results validated the effectiveness of the meta-learning approach on the VLN-CE dataset.
• MAML and the metric-based method demonstrated distinct advantages in seen and unseen environments, respectively.

Conclusions:

• Meta-learning provides an effective solution for visual perception generalization in VLN agents.
• The strategy allows for efficient adaptation of navigation skills across diverse robotic camera configurations.
• The study highlights the potential for improved robot navigation through adaptable visual perception systems.