Multimodal Representation Learning for Place Recognition Using Deep Hebbian Predictive Coding.

Area of Science:

• Robotics
• Neuroscience
• Artificial Intelligence

Background:

• Multisensory place recognition in robotics faces challenges like data mismatch and sensor dropout.
• Animals exhibit robust navigation, offering inspiration for bio-plausible AI solutions.
• Current methods for multisensory data fusion are often brittle and lack adaptability.

Purpose of the Study:

• To develop a neuro-ethological approach for robust multisensory place recognition in robots.
• To leverage self-supervised representation learning based on predictive coding for robot navigation.
• To enhance robot navigation accuracy and resilience in complex environments.

Main Methods:

• Utilized a neuroscientific model of the visual cortex, predictive coding, for representation learning.
• Developed a parsimonious network algorithm with a local learning rule.
• Extended the algorithm to fuse visual and tactile sensory cues from a biomimetic robot.

Main Results:

• Achieved significantly better place recognition performance using joint latent representations compared to contemporary methods.
• Demonstrated improved robustness in place recognition despite unimodal sensor drop-out.
• The proposed multimodal deep predictive coding algorithm is linearly extensible to more sensory modalities.

Conclusions:

• Neuro-biologically plausible representation learning offers significant advantages for multimodal navigation.
• The deep predictive coding algorithm provides a robust and extensible solution for multisensory fusion in robotics.
• This approach enhances robot navigation capabilities by mimicking natural sensory processing.