TerrainFormer: World Model-Guided Decision Transformer for Autonomous Off-Road Navigation.

Area of Science:

• Robotics and Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Autonomous navigation in unstructured off-road environments is challenging due to complex terrain and lack of road markings.
• Real-time traversability reasoning from raw sensory data is crucial for safe navigation.

Purpose of the Study:

• To develop a hierarchical framework, TerrainFormer, for robust autonomous navigation in off-road environments.
• To enable effective cross-dataset generalization for terrain prediction and action selection.

Main Methods:

• TerrainFormer integrates a world model for terrain dynamics prediction and a temporal decision transformer for action selection.
• A two-phase training paradigm involves self-supervised world model pretraining on LiDAR data and behavioral cloning of the decision transformer.
• The world model utilizes PointPillars and a Vision Transformer for real-time Bird's-Eye-View (BEV) projection and terrain representation.

Main Results:

• Achieved 87.31% test accuracy and 0.7948 macro F1 score on the RELLIS-3D dataset across 12 action classes.
• The world model demonstrated high fidelity in predicting future frames, with only a 0.79% accuracy drop compared to ground truth.
• Showcased 98.82% action agreement, highlighting effective cross-dataset generalization.

Conclusions:

• TerrainFormer provides an effective solution for real-time autonomous navigation in challenging off-road conditions.
• The proposed cross-dataset generalization paradigm allows the model to perform well without overlapping training data.
• The framework successfully learns directionally conditioned navigation policies and addresses data imbalance issues.