Learning Selective Sensor Fusion for State Estimation.

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Autonomous vehicles and mobile robots rely on multiple sensors for environmental perception and state estimation.
• Current deep learning (DL) models for sensor fusion often lack robustness to real-world noisy or incomplete data and suffer from poor interpretability.

Purpose of the Study:

• To introduce SelectFusion, an end-to-end selective sensor fusion module for robust multimodal odometry and localization.
• To develop a uniform DL framework for sensor fusion applicable to various modalities and tasks, enhancing interpretability.

Main Methods:

• Proposed SelectFusion, a DL module for selective sensor fusion, applicable to modalities like images, inertial measurements, depth, and LIDAR.
• Introduced deterministic soft fusion and stochastic hard fusion modules, comparing them against direct fusion strategies.
• Evaluated fusion strategies on public and degraded datasets with occlusions, noise, missing data, and time misalignment.

Main Results:

• SelectFusion effectively assesses feature reliability from different sensors, enabling accurate trajectory estimation at both scale and global pose.
• The proposed fusion strategies demonstrated superior performance compared to direct fusion, especially under challenging data conditions.
• Analysis provided insights into how fusion strategies attend to reliable features, enhancing model interpretability.

Conclusions:

• SelectFusion offers a robust and interpretable solution for sensor fusion in autonomous systems, outperforming traditional methods.
• The framework's flexibility allows adaptation to diverse sensor modalities and robotic applications.
• The study highlights the importance of selective fusion for reliable state estimation in real-world autonomous systems.