Curriculum Reinforcement Learning Based on K-Fold Cross Validation.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Intelligent Control Systems

Background:

• Deep reinforcement learning (DRL) is advancing intelligent control.
• Combining automatic curriculum learning (ACL) with DRL can boost algorithm performance and efficiency.
• Existing ACL methods struggle with task ranking and slow convergence due to reliance on expert experience and single networks.

Purpose of the Study:

• To propose a novel curriculum reinforcement learning method using K-Fold Cross Validation.
• To estimate the relative difficulty score of curriculum tasks.
• To improve the training speed and efficiency of multi-agent deep reinforcement learning algorithms.

Main Methods:

• A K-Fold Cross Validation-based curriculum reinforcement learning approach is introduced.
• The method divides ACL into distinct difficulty assessment and sorting stages.
• Parallel training of a teacher model and cross-evaluation of task sample difficulty are employed for task sequencing.

Main Results:

• The proposed method effectively estimates the relative difficulty of curriculum tasks.
• Simulation experiments in multi-agent environments demonstrate improved training speed for the MADDPG algorithm.
• The approach shows generality for multi-agent DRL algorithms utilizing replay buffer mechanisms.

Conclusions:

• The K-Fold Cross Validation-based ACL method enhances the training efficiency of multi-agent DRL.
• This approach offers a more effective strategy for sequencing curriculum learning tasks compared to existing methods.
• The proposed method provides a scalable and efficient solution for complex DRL training scenarios.