Adaptive Cross-Modal Denoising: Enhancing LiDAR-Camera Fusion Perception in Adverse Circumstances

Area of Science:

• Robotics and Artificial Intelligence
• Sensor Fusion and Perception
• Computer Vision

Background:

• Autonomous vehicles (AVs) depend on LiDAR and camera sensors for environmental perception.
• Adverse weather conditions like rain, snow, and fog degrade sensor performance, introducing noise and reducing reliability.
• Current denoising methods often focus on single sensor types, failing to exploit complementary multimodal data.

Purpose of the Study:

• To develop an advanced denoising framework for multimodal sensor data in adverse weather.
• To improve the reliability and safety of autonomous vehicles under challenging environmental conditions.
• To leverage the synergistic relationship between LiDAR and camera data for enhanced perception.

Main Methods:

• Proposed a novel Adaptive Cross-Modal Denoising (ACMD) framework.
• Utilized modality-specific self-denoising encoders and an Adaptive Bridge Controller (ABC) for noise evaluation and cross-modal guidance.
• Incorporated a Cross-Modal Denoising (CMD) module for semantic refinement of noisy sensor data.

Main Results:

• ACMD significantly outperformed unimodal denoising methods (Restormer, PathNet, BM3D, PointCleanNet) by 28.2% in PSNR and 33.3% in CD.
• ACMD achieved a 16.2% improvement in JDE compared to state-of-the-art fusion models.
• Experimental validation was conducted on the WeatherKITTI dataset using synthetically added noise.

Conclusions:

• The ACMD framework effectively addresses multimodal sensor noise in adverse weather.
• Enhanced sensor data quality directly contributes to improved AV reliability and operational safety.
• This approach paves the way for more robust autonomous driving systems in diverse environmental conditions.