Structured Cooperative Reinforcement Learning With Time-Varying Composite Action Space.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Reinforcement learning excels in static, low-dimensional action spaces.
• Practical tasks feature composite, time-varying action spaces with dynamic sub-actions.
• Existing methods struggle with robustness and transferability in such environments.

Purpose of the Study:

• To develop a reinforcement learning algorithm for robust and transferable control in time-varying composite action spaces.
• To address challenges posed by invalid or newly added sub-actions.
• To improve cooperative learning in complex, dynamic environments.

Main Methods:

• Proposes SCORE (Structured Cooperative Reinforcement Learning algorithm).
• Models problems as fully cooperative partially observable stochastic games.
• Utilizes graph attention networks for sub-action dependency modeling.
• Employs a hierarchical variational autoencoder for a common latent action space.
• Incorporates implicit credit assignment for multi-agent challenges.

Main Results:

• SCORE demonstrates significant advantages in robustness.
• SCORE shows improved transferability in time-varying composite action spaces.
• Experiments conducted on proof-of-concept and precision agriculture tasks.

Conclusions:

• SCORE effectively handles the complexities of time-varying composite action spaces.
• The algorithm offers a robust and transferable solution for dynamic reinforcement learning problems.
• This work advances reinforcement learning applications in complex real-world scenarios.