PVNAS: 3D Neural Architecture Search With Point-Voxel Convolution

Area of Science:

• Computer Vision
• Deep Learning
• Hardware-Efficient AI

Background:

• 3D neural networks are crucial for applications like AR/VR and autonomous driving.
• Existing voxel-based and point-based methods face hardware limitations due to memory footprint and random access.
• Efficiency is a key challenge for deploying 3D deep learning on resource-constrained edge devices.

Purpose of the Study:

• To analyze performance bottlenecks in current 3D deep learning methods.
• To propose a novel, hardware-efficient 3D primitive for improved efficiency.
• To develop an automated method for designing resource-constrained 3D network architectures.

Main Methods:

• Developed Point-Voxel Convolution (PVConv), a hybrid primitive combining point-based and voxel-based approaches.
• Enhanced PVConv with sparse convolution for processing large-scale outdoor scenes.
• Introduced 3D Neural Architecture Search (3D-NAS) to optimize network architectures under resource constraints.

Main Results:

• Achieved state-of-the-art performance on six benchmark datasets.
• Demonstrated significant speedups ranging from 1.8x to 23.7x.
• Successfully deployed on MIT Driverless racing vehicles, improving detection range, accuracy, and reducing latency.

Conclusions:

• PVConv offers a hardware-efficient solution for 3D deep learning on edge devices.
• 3D-NAS effectively finds optimal architectures for resource-constrained environments.
• The proposed methods show practical benefits in real-world autonomous systems.