Brain-inspired multimodal hybrid neural network for robot place recognition.

Area of Science:

• Robotics and Artificial Intelligence
• Computational Neuroscience
• Sensor Fusion

Background:

• Place recognition is crucial for robot navigation but challenging in natural, dynamic environments.
• Humans and animals exhibit robust place recognition despite environmental variations.
• Existing robot systems struggle with resource limitations and environmental changes.

Purpose of the Study:

• To develop a general place recognition (GPR) system inspired by biological neural mechanisms.
• To enhance robot navigation capabilities in complex and unpredictable environments.
• To leverage multimodal sensing and bio-inspired computing for robust spatial intelligence.

Main Methods:

• A multimodal hybrid neural network (MHNN) was designed to integrate sensory cues.
• Specialized neural networks (spatial view, place, head direction, time cells) were implemented for encoding.
• A multiscale liquid state machine processed and fused multimodal information asynchronously.
• The MHNN was deployed on the Tianjic hybrid neuromorphic chip and integrated into a quadruped robot.

Main Results:

• NeuroGPR demonstrated superior performance compared to conventional and existing biologically inspired methods.
• The system exhibited robustness against perceptual aliasing, motion blur, and environmental changes (light, weather).
• On the Tianjic chip, NeuroGPR achieved 10.5x lower latency and 43.6% lower power consumption than Jetson Xavier NX.

Conclusions:

• NeuroGPR offers a powerful, bio-inspired solution for general place recognition in robots.
• The system effectively mimics neural mechanisms for robust and efficient spatial intelligence.
• The use of hybrid neuromorphic chips significantly enhances the efficiency of GPR systems.