Representation learning for continuous action spaces is beneficial for efficient policy learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Deep reinforcement learning (DRL) integrates deep learning's perception with traditional reinforcement learning (RL).
• Model-free RL methods learn state representations and policies end-to-end, but struggle with large state/action spaces, requiring extensive data and time.
• Existing DRL policies often exhibit poor generalization, particularly in continuous action spaces.

Purpose of the Study:

• To propose an efficient policy learning method for DRL that enhances generalization in large-scale continuous action and state spaces.
• To address the limitations of traditional DRL methods regarding sample efficiency and policy generalization.
• To develop a method that leverages latent representations for both states and actions.

Main Methods:

• The proposed method learns policy in latent state and action spaces, extending representation learning to actions.
• The learning process is divided into unsupervised learning of large-scale representation models and RL-based learning of a small-scale policy model.
• This approach decouples representation learning from policy optimization for improved efficiency and generalization.

Main Results:

• The method demonstrates effectiveness in improving policy generalization across various continuous action spaces.
• It reduces the sample complexity and training time compared to traditional end-to-end DRL methods.
• Experiments on MountainCar, CarRacing, and Cheetah validate the proposed approach's performance.

Conclusions:

• The proposed latent space policy learning method offers an efficient and effective solution for DRL challenges in large-scale continuous control tasks.
• Extending representation learning to action spaces significantly boosts policy generalization capabilities.
• This approach provides a scalable and robust framework for real-world DRL applications.