Automated vehicle data pipeline for accident reconstruction: New insights from LiDAR, camera, and radar data

As automated vehicles are deployed across the world, it has become critically important to understand how these vehicles interact with each other, as well as with other conventional vehicles on the road. One such method to achieve a deeper understanding of the safety implications for Automated Vehicles (AVs) is to analyze instances where AVs were involved in crashes. Unfortunately, this poses a steep challenge to crash-scene investigators. It is virtually impossible to fully understand the factors that contributed to an AV involved crash without taking into account the vehicle's perception and decision making. Furthermore, there is a tremendous amount of data that could provide insight into these crashes that is currently unused, as it also requires a deep understanding of the sensors and data management of the vehicle. To alleviate these problems, we propose a data pipeline that takes raw data from all on-board AV sensors such as LiDAR, radar, cameras, IMU's, and GPS's. We process this data into visual results that can be analyzed by crash scene investigators with no underlying knowledge of the vehicle's perception system. To demonstrate the utility of this pipeline, we first analyze the latest information on AV crashes that have occurred in California and then select two crash scenarios that are analyzed in-depth using high-fidelity synthetic data generated from the automated vehicle simulator CARLA. The data visualization procedure is demonstrated on the real-world Kitti dataset by using the YOLO object detector and a monocular depth estimator called AdaBins. Depth from LIDAR is used as ground truth to calibrate and assess the effect of noise and errors in depth estimation. The visualization and data analysis from these scenarios clearly demonstrate the vast improvement in crash investigations that can be obtained from utilizing state-of-the-art sensing and perception systems used on AVs.