RGB-to-Infrared Translation Using Ensemble Learning Applied to Driving Scenarios

Area of Science:

• Computer Vision
• Autonomous Systems
• Sensor Fusion

Background:

• Robust perception is crucial for autonomous vehicles.
• Infrared (IR) imaging complements RGB sensors but lacks sufficient data in existing datasets.
• Limited IR data hinders the development of advanced sensor fusion algorithms.

Purpose of the Study:

• To develop a method for synthesizing realistic IR images from RGB data.
• To address the scarcity of IR data in datasets and simulation tools.
• To enhance the robustness of autonomous vehicle perception systems through improved sensor fusion.

Main Methods:

• Utilized gradient-boosting decision trees for image translation.
• Synthesized Near-Infrared (NIR) and thermal images from RGB images.
• Applied the method to real-world datasets (MS2, EPFL, Freiburg) and synthetic CARLA data.

Main Results:

• Achieved high-quality synthetic IR image generation, outperforming traditional deep learning methods in data-scarce scenarios.
• Validated synthetic image quality using R2 (0.98 on MS2), PSNR (21.3 dB on Freiburg), SSIM, and LPIPS metrics.
• Demonstrated effective enhancement of existing datasets and synthetic driving scenarios.

Conclusions:

• The proposed method provides a viable alternative to traditional deep learning for IR image synthesis.
• Enriched datasets with synthetic IR data improve sensor fusion algorithm development and validation.
• Facilitates the creation of diverse driving scenarios to enhance autonomous driving system robustness.