DynamicVLN: Incorporating Dynamics into Vision-and-Language Navigation Scenarios

Area of Science:

• Robotics
• Artificial Intelligence
• Computer Vision

Background:

• Traditional Vision-and-Language Navigation (VLN) models struggle with dynamic real-world environments.
• Static environments do not reflect complexities like moving vehicles, pedestrians, and changing traffic signals.

Purpose of the Study:

• Introduce Dynamic Vision-and-Language Navigation (DynamicVLN) to address limitations of static VLN tasks.
• Enhance agent decision-making and adaptability in dynamic navigation scenarios.

Main Methods:

• Developed ten dynamic navigation scenarios.
• Created a dataset of 11,261 instances using the CARLA simulator and large language models.
• Introduced a baseline model with advanced perception and decision-making modules.

Main Results:

• The baseline model effectively navigates and interprets dynamic road conditions.
• Demonstrated agent ability to follow instructions and adapt to environmental changes.
• Established a benchmark for dynamic navigation agents.

Conclusions:

• DynamicVLN is crucial for developing agents capable of real-world navigation.
• The proposed approach moves beyond static VLN limitations towards practical applications.
• Agents need reasoning and adaptability for unforeseen events in dynamic environments.