Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation.

Area of Science:

• Artificial Intelligence
• Robotics
• Natural Language Processing

Background:

• Natural language grounded navigation tasks heavily rely on language instructions.
• Current navigators struggle with complex instructions, leading to performance issues.
• Limited human-annotated data hinders the ability to capture crucial navigational information.

Purpose of the Study:

• To develop a more robust navigator capable of dynamically extracting key information from lengthy instructions.
• To enhance navigator performance in complex, language-grounded navigation scenarios.
• To address the limitations of navigators trained on insufficient or overly simplistic instructions.

Main Methods:

• Proposed a Dynamic Reinforced Instruction Attacker (DR-Attacker) using an adversarial attacking paradigm.
• Formulated instruction perturbation generation as a Markov Decision Process, optimized via reinforcement learning.
• Employed adversarial training and a self-supervised reasoning task with perturbed instructions as hard samples.

Main Results:

• Demonstrated superior performance over state-of-the-art methods on Vision-and-Language Navigation (VLN) and Navigation from Dialog History (NDH) tasks.
• Validated the effectiveness of DR-Attacker in identifying and attacking crucial instruction information at different timesteps.
• Achieved improved navigator robustness through adversarial training with challenging, machine-generated instructions.

Conclusions:

• The proposed DR-Attacker significantly enhances the robustness of language-grounded navigation systems.
• Adversarial training with dynamically generated perturbed instructions is an effective strategy for improving navigator performance.
• The method shows promise for real-world applications requiring reliable navigation based on complex natural language.