ConvNet-Generated Adversarial Perturbations for Evaluating 3D Object Detection Robustness.

This research develops a novel adversarial Convolutional Neural Network (ConvNet) to generate perturbations in 3D point clouds for rapid robustness testing. The investigators seek to embed this generative model directly into the detection pipeline at the voxel feature level rather than the raw point level. By training the network to maximize detection loss, the team aims to identify specific weaknesses in LiDAR-based perception systems that might be missed by random noise. The study intends to ensure that generated adversarial noise remains within realistic sensor error bounds to simulate real-world conditions effectively. Researchers focus on creating a system that allows for one-time training followed by fast, inference-time evaluation to support real-time applications. The work targets the identification of vulnerability patterns across different object classes and sizes in urban driving scenarios to improve safety protocols. This effort addresses the need for scalable security audits of complex 3D vision models used in safety-critical transportation.

The experimental framework utilizes a Sparsely Embedded Convolutional Detection (SECOND) detector and the CenterPoint architecture for performance benchmarking. Data for training and evaluation are sourced from the KITTI and NuScenes datasets, which provide diverse 3D point cloud environments and varied object distributions. The proposed ConvNet architecture incorporates multi-component loss constraints, including intensity, bias, and imbalance terms, to regulate the physical characteristics of the noise. These constraints ensure that the adversarial modifications do not exceed typical sensor error margins of 0.9 to 2.6 centimeters at standard urban distances. The team measures the impact of the perturbations by calculating the mean Average Precision (mAP) and weighted mAP across various object categories to quantify performance drops. Computational efficiency is assessed by timing the forward pass overhead during the inference phase of the detection pipeline to ensure minimal latency. The researchers compare the effectiveness of their gradient-free approach against traditional iterative methods to validate the efficiency gains.

CenterPoint on the NuScenes dataset exhibits a 24% weighted mAP reduction across ten distinct object classes when subjected to the generated noise. The SECOND detector on the KITTI dataset shows an 8% overall mAP degradation under similar adversarial conditions, indicating varying levels of model robustness. Analysis of object categories reveals that smaller entities like pedestrians and cyclists suffer higher vulnerability rates of 13% and 14%, respectively, compared to larger targets. Larger vehicles such as cars demonstrate significant resilience, showing only a 0.2% decrease in detection accuracy on the KITTI benchmark despite the perturbations. Barriers and pedestrians on the NuScenes platform are the most affected groups, with barriers experiencing a 68% drop in detection performance during the evaluation. The adversarial perturbations maintain a mean L2 norm of 0.09% for KITTI and 0.05% for NuScenes, keeping them within standard sensor noise levels. The system achieves a remarkably low overhead of 1.2 to 2.0 milliseconds per frame, making it practical for integration into live autonomous systems.

The findings suggest that adversarial 3D perturbations can cause substantial detection failures even when they remain within typical sensor error margins. This research highlights a critical inverse relationship between the physical size of an object and its susceptibility to adversarial attacks in point cloud data. The low computational overhead of 1.2 to 2.0 milliseconds makes this approach suitable for real-time safety monitoring in autonomous vehicles during operation. Implementing this gradient-free evaluation method allows developers to continuously assess the robustness of perception systems without the need for expensive iterative computations. Future safety standards for self-driving technology may need to incorporate these generative models to ensure reliable obstacle detection under diverse environmental conditions. The study provides a practical framework for identifying and mitigating specific vulnerabilities in LiDAR-based object detection pipelines across multiple datasets. These results emphasize the necessity of considering adversarial robustness as a primary metric in the development of 3D vision systems for transportation.