Curriculum reinforcement learning with measurable task representation learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Curriculum Reinforcement Learning (CRL) agents learn from a sequence of tasks to solve a target task.
• Automatic curriculum generation is an emerging area in CRL, moving beyond manual task sequencing.
• Existing interpolation-based CRL methods struggle with non-Euclidean task spaces common in navigation.

Purpose of the Study:

• To develop a novel automatic curriculum generation approach for complex navigation tasks.
• To address the limitations of interpolation-based methods in non-Euclidean task spaces.
• To enable effective learning by generating intermediate tasks that are progressively similar to the target task.

Main Methods:

• Proposed a novel automatic curriculum generation approach based on measurable task representation learning.
• Transformed the task space into a latent space using a variational autoencoder (VAE).
• The VAE encodes reward and state transitions to learn a latent task representation with similarity properties.

Main Results:

• The learned latent task representation effectively measures task similarity based on rewards and state transitions.
• The developed curriculum generation scheme successfully creates tasks increasingly similar to the target task.
• Experimental results on challenging navigation tasks show the proposed method outperforms state-of-the-art CRL approaches.

Conclusions:

• The proposed measurable task representation learning approach enables effective automatic curriculum generation in complex, non-Euclidean task spaces.
• This method offers a significant advancement over existing interpolation and generative adversarial network-based CRL techniques.
• The approach holds promise for improving agent performance in challenging reinforcement learning environments.