Balancing State Exploration and Skill Diversity in Unsupervised Skill Discovery.

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Unsupervised skill discovery aims to acquire useful skills without external rewards using unsupervised reinforcement learning (RL).
• Current advanced methods face challenges in balancing state exploration and skill diversity, especially with complex skill sets.
• This limitation hinders the acquisition of robust and adaptable skills for various downstream applications.

Purpose of the Study:

• To propose Contrastive Dynamic Skill Discovery (ComSD), a novel approach for generating diverse and exploratory unsupervised skills.
• To address the limitations of existing methods in balancing state exploration and skill diversity.
• To enhance the adaptability of discovered skills for downstream tasks.

Main Methods:

• Introduced a novel intrinsic incentive: contrastive dynamic reward.
• Incorporated a particle-based exploration reward to encourage access to far-reaching states.
• Developed a contrastive diversity reward to enhance skill discriminability.
• Implemented a dynamic weighting mechanism to balance exploration and diversity rewards.

Main Results:

• ComSD successfully generated diverse behaviors with varying exploratory levels in multijoint robots.
• The method demonstrated state-of-the-art adaptation performance on challenging downstream tasks.
• ComSD discovered distinguishable and far-reaching exploration skills in a complex 2-D maze environment.

Conclusions:

• ComSD effectively generates diverse and exploratory unsupervised skills by optimizing the balance between state exploration and skill diversity.
• The proposed method significantly improves the adaptability of learned skills for complex robotic tasks.
• ComSD offers a promising direction for advancing unsupervised skill discovery in reinforcement learning.