An Embodied Multi-Sensor Fusion Approach to Visual Motion Estimation Using Unsupervised Deep Networks

Area of Science:

• Robotics
• Computer Vision
• Deep Learning

Background:

• Robotic vision-aided state estimation is crucial for autonomous systems.
• Size, Weight, and Power (SWaP)-constrained environments pose significant challenges.
• Existing methods struggle with noisy and heterogeneous sensor data.

Purpose of the Study:

• To improve SWaP-constrained robotic vision-aided state estimation.
• To introduce an unsupervised deep learning network for robust sensor fusion.
• To enhance the accuracy and efficiency of image correspondence prediction.

Main Methods:

• Developed Multi-Hypothesis DeepEfference (MHDE), an unsupervised deep convolutional-deconvolutional sensor fusion network.
• MHDE intelligently combines noisy heterogeneous sensor data using parallel architectural pathways and multi-hypothesis sub-pathways.
• Evaluated MHDE on the KITTI Odometry dataset.

Main Results:

• MHDE demonstrated increased robustness against dynamic, heteroscedastic sensor and motion noise.
• Achieved high-speed hypothesis image mappings and predictions (76–357 Hz).
• Outperformed vision-only algorithms like DeepMatching and Deformable Spatial Pyramids in runtime and performance.

Conclusions:

• MHDE offers a significant advancement in robotic state estimation for SWaP-constrained applications.
• The multi-hypothesis approach enhances resilience to sensor noise and motion artifacts.
• MHDE provides a more efficient and accurate solution for dense pixel-level image correspondence.