Off-Road Autonomous Vehicle Semantic Segmentation and Spatial Overlay Video Assembly

Area of Science:

• Robotics and Artificial Intelligence
• Computer Vision
• Data Compression

Background:

• Autonomous systems require robust perception for unstructured environments.
• Off-road autonomy faces challenges like dynamic terrain and limited communication.
• Existing solutions struggle with the complexity of off-road navigation.

Purpose of the Study:

• To develop a comprehensive solution for off-road autonomous vehicle perception and operation.
• To address the unique challenges posed by unstructured, off-road environments.
• To enable efficient real-time remote operation in bandwidth-constrained scenarios.

Main Methods:

• Curated a large-scale off-road dataset for realistic training and model generalization.
• Proposed a Confusion-Aware Loss (CAL) to improve semantic segmentation accuracy.
• Introduced a spatial overlay video encoding scheme for efficient data transmission.

Main Results:

• CAL improved segmentation mIoU from 68.66% to 70.06% on off-road data.
• Achieved cross-domain mIoU gains of up to 0.49% on the Cityscapes dataset.
• Video encoding improved PSNR by up to +5 dB and VMAF by up to +40 points.

Conclusions:

• The integrated three-part solution enhances off-road autonomous vehicle perception and efficiency.
• The developed methods provide a robust framework for challenging, unstructured environments.
• This research facilitates reliable real-time remote operation under bandwidth constraints.