Model-Based Reinforcement Learning With Isolated Imaginations.

Area of Science:

• Artificial Intelligence
• Robotics
• Machine Learning

Background:

• World models are crucial for vision-based interactive systems but struggle with noncontrollable dynamics.
• Autonomous driving presents challenges due to independent or sparsely dependent environmental variations.
• Existing models face difficulties in learning effective world representations in such complex scenarios.

Purpose of the Study:

• To propose Iso-Dream++, a novel model-based reinforcement learning approach.
• To enhance world models by isolating controllable state transitions from mixed environmental variations.
• To improve long-horizon visuomotor control by decoupling latent imaginations.

Main Methods:

• Optimizing inverse dynamics to separate controllable from noncontrollable state transitions.
• Performing policy optimization using decoupled latent imaginations.
• Rolling out noncontrollable states and adaptively associating them with controllable states.
• Addressing sparse dependencies and training collapse in state decoupling.

Main Results:

• Iso-Dream++ effectively isolates controllable dynamics from mixed spatiotemporal variations.
• The approach enables effective long-horizon visuomotor control by leveraging decoupled imaginations.
• Significant performance improvements were observed compared to existing reinforcement learning models.
• Validation in transfer learning setups demonstrated the approach's robustness.

Conclusions:

• Iso-Dream++ offers a robust solution for learning world models in interactive systems with noncontrollable dynamics.
• The method enhances autonomous driving capabilities by enabling better anticipation of environmental changes.
• This work advances reinforcement learning for complex, real-world applications.