Deep reinforcement learning navigation via decision transformer in autonomous driving

Area of Science:

• Robotics
• Artificial Intelligence
• Computer Vision

Background:

• Autonomous driving decisions are sequential, based on partial, noisy environmental observations.
• Reinforcement learning (RL) is commonly used to train agents with rewards from limited data.
• Urban autonomous navigation faces challenges due to partial observability and unknown environment models.

Purpose of the Study:

• Introduce a novel algorithm, deep reinforcement learning navigation via decision transformer (DRLNDT), to improve autonomous vehicle decision-making.
• Enhance the capabilities of autonomous vehicles in partially observable urban settings.
• Address challenges posed by sensor noise and occlusions in real-world driving scenarios.

Main Methods:

• The DRLNDT framework utilizes the Soft Actor-Critic (SAC) algorithm.
• Transformer neural networks model temporal dependencies in observations and actions.
• A variational autoencoder (VAE) extracts latent vectors from images, reducing state-space dimensionality and improving training efficiency.

Main Results:

• DRLNDT effectively mitigates judgment errors caused by sensor noise or occlusion.
• The multimodal state space integrates vector states (velocity, position) and image-derived latent vectors.
• Experiments show DRLNDT achieves a superior optimal policy without prior environmental knowledge, maps, or routing assistance.

Conclusions:

• DRLNDT surpasses baseline and other policy methods lacking historical data.
• The algorithm demonstrates enhanced decision-making for autonomous vehicles in complex urban environments.
• DRLNDT offers a robust approach to navigation in partially observable conditions.