3D object detection through fog and occlusion: passive integral imaging vs active (LiDAR) sensing

Area of Science:

• Computer Vision
• Sensor Fusion
• 3D Imaging

Background:

• Object recognition systems struggle in degraded environments like fog and partial occlusion.
• Existing systems using Long Wave Infrared (LWIR) and LiDAR (Azure Kinect) have limitations in adverse weather and low spatial resolution.
• Visible, LWIR, and LiDAR sensors are affected differently by environmental degradation, impacting object detection performance.

Purpose of the Study:

• To evaluate the effectiveness of a 3D integral imaging (InIm) system using visible range cameras for object recognition in degraded environments.
• To compare the performance of 3D InIm against LWIR and Azure Kinect depth imaging systems under fog and occlusion.
• To investigate the potential of 3D InIm to mitigate challenges posed by fog and occlusion in object detection tasks.

Main Methods:

• Utilized a 3D integral imaging (InIm) system with a visible range camera.
• Employed the YOLOv3 neural network for object recognition across visible, LWIR, and Azure Kinect depth imaging systems.
• Compared system performance using metrics such as average precision and average miss rate in controlled degraded environments.

Main Results:

• 3D integral imaging with visible cameras demonstrated superior image reconstruction in degraded environments compared to LWIR and Azure Kinect.
• The 3D InIm system showed improved object detection accuracy in fog and partial occlusion.
• This study provides the first comparison of passive integral imaging versus active (LiDAR) sensing for object detection in degraded conditions.

Conclusions:

• 3D integral imaging using visible range cameras offers a promising solution for enhancing object recognition in challenging environments.
• The findings suggest that 3D InIm can overcome the limitations of LWIR and LiDAR-based systems in adverse weather.
• Further research can explore the integration of 3D InIm with other sensing modalities for even more robust object detection.