Behind-The-Scenes (BTS): Wiper-Occlusion Canceling for Advanced Driver Assistance Systems in Adverse Rain...

Area of Science:

• Computer Vision
• Automotive Engineering

Background:

• Vision-based Advanced Driver Assistance Systems (ADAS) face challenges in adverse rainy conditions.
• Windshield wipers, while clearing raindrops, create occlusions that degrade image quality and hinder ADAS performance.
• Wiper-induced occlusions can lead to erroneous judgments and compromise vehicle safety.

Purpose of the Study:

• To propose and evaluate a real-time system, Behind-the-Scenes (BTS), for detecting and removing wiper-occlusion in images captured under rainy weather.
• To enhance the reliability and accuracy of vision-based ADAS in challenging driving environments.

Main Methods:

• Developed BTS to detect pixel-wise wiper masks using high-pass filtering and optical flow prediction from sequential image pairs.
• Fine-tuned a deep learning optical flow model using a synthesized dataset with auto-labeled wiper masks and flows from real rainy images.
• Enhanced dataset diversity by synthesizing static objects with real fast-moving objects and annotating real images for ground truth evaluation.

Main Results:

• BTS achieved a 0.962 SSIM and 91.6% F1 score for wiper mask detection.
• Achieved an 88.3% F1 score for wiper image detection.
• Demonstrated significant improvement in vision-based image restoration and object detection applications by effectively canceling occlusions.

Conclusions:

• BTS effectively detects and removes windshield wiper occlusion in real-time under rainy conditions.
• The proposed method significantly enhances the performance of vision-based applications crucial for ADAS.
• BTS shows strong potential for improving overall ADAS safety and functionality in adverse weather.