A Comparative Study and Optimization of Camera-Based BEV Segmentation for Real-Time Autonomous Driving.

Area of Science:

• Computer Vision
• Autonomous Driving Systems
• Embedded Systems Engineering

Background:

• Real-time bird's eye view (BEV) segmentation is crucial for autonomous driving safety.
• Existing methods face challenges in balancing accuracy and computational efficiency on embedded platforms.
• Optimization of camera-based BEV segmentation for resource-constrained environments is a key research area.

Purpose of the Study:

• To optimize a camera-based bird's eye view (BEV) segmentation technique for real-time embedded system deployment.
• To evaluate and compare depth-based, MLP-based, and transformer-based BEV segmentation methods.
• To achieve high accuracy and low latency through a multi-stage optimization process.

Main Methods:

• Mathematical analysis and comparative performance evaluation of lift-splat-shoot, HDMapNet, and BEVFormer on the nuScenes dataset.
• Three-stage optimization: accuracy improvement (module selection, input resolution, data augmentation), latency reduction, and model size optimization (model compression).
• Utilized InternImage-B/T encoders, EfficientNet-B0 decoder, and FP16 quantization for model compression.

Main Results:

• The lift-splat-shoot method with InternImage-B encoder and EfficientNet-B0 decoder achieved 54.9 mIoU at 448x800 input resolution.
• The InternImage-T variant offered high efficiency (51.7 ms latency, 159.5 MB size) with 53.1 mIoU.
• FP16 quantization reduced memory by 50% and latency, maintaining mIoU. The optimized method improved mIoU by 29.2% with reduced memory size.

Conclusions:

• The InternImage encoder-based lift-splat-shoot technique provides the best trade-off between accuracy, latency, and model size for embedded BEV segmentation.
• Optimal input resolution is model-dependent, requiring careful tuning for maximum accuracy.
• Model compression techniques like FP16 quantization are effective for deploying accurate BEV segmentation on power-constrained embedded devices.