A Novel Multi-Sensor Nonlinear Tightly-Coupled Framework for Composite Robot Localization and Mapping

Area of Science:

• Robotics
• Computer Vision
• Sensor Fusion

Background:

• Composite robots face challenges in perception and pose estimation due to illumination changes, disturbances, and sensor errors.
• Existing systems struggle with accuracy and reliability in dynamic and low-light environments.

Purpose of the Study:

• To develop an integrated localization and navigation framework, IIVL-LM, that overcomes environmental perception and pose estimation difficulties.
• To enhance the robustness and precision of robot systems in complex, variable conditions.

Main Methods:

• Proposed a nonlinear optimization approach for tightly coupled data-level fusion of IMU, infrared, RGB camera, and LiDAR data.
• Developed a real-time luminance calculation model and a fast approximation method for feature fusion.
• Optimized the Visual-Inertial Odometry (VIO) module using infrared camera depth information within the R3LIVE++ framework.

Main Results:

• The IIVL-LM system demonstrated significant performance improvements in challenging luminance conditions, particularly in low-light environments.
• Achieved an average RMSE ATE improvement of 23% to 39% (0.006 to 0.013) in simulated indoor rescue scenarios.
• Verified the critical importance of infrared image fusion through comparative experiments on the TUM-VI dataset.

Conclusions:

• The IIVL-LM framework significantly boosts robot robustness and precision in unknown and expansive environments by maintaining active engagement of at least three sensors.
• This integrated approach is crucial for applications requiring high reliability in complex scenarios, such as indoor rescue operations.