Multimodal information bottleneck for deep reinforcement learning with multiple sensors.

Area of Science:

• Robotics
• Machine Learning
• Computer Vision

Background:

• Reinforcement learning (RL) shows promise in robotics but struggles with multimodal sensory input.
• Current methods use auxiliary losses for joint representations, yet may include irrelevant information, hindering performance.
• Effective fusion of diverse sensory data is crucial for enhancing RL sample efficiency and policy learning.

Purpose of the Study:

• To develop a multimodal information bottleneck model for learning task-relevant joint representations from egocentric images and proprioception.
• To improve reinforcement learning performance in robotic control by effectively integrating complementary sensory information.
• To filter out task-irrelevant information from raw multimodal observations.

Main Methods:

• Proposed a multimodal information bottleneck (MIB) model to learn compressed, task-relevant joint representations.
• MIB compresses and retains predictive information from multimodal observations (images and proprioception).
• Optimized the model by minimizing the upper bound of the MIB objective for computational tractability.

Main Results:

• The MIB model demonstrated superior sample efficiency in challenging robotic locomotion tasks.
• Achieved significant zero-shot robustness to unseen white noise compared to leading baselines.
• Empirically validated that combining egocentric images and proprioception is more beneficial than using single modalities for locomotion policies.

Conclusions:

• The proposed multimodal information bottleneck effectively learns compressed, task-relevant representations from egocentric images and proprioception.
• This approach enhances reinforcement learning for robotic control, improving sample efficiency and robustness.
• Leveraging complementary information from multiple sensory modalities is key to advancing robotic locomotion capabilities.