Generative subgoal oriented multi-agent reinforcement learning through potential field.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Robotics

Background:

• Multi-agent reinforcement learning (MARL) utilizes subgoals to accelerate agent learning in sparse reward environments.
• Existing MARL methods often lack consistency between subgoal generation and achievement, hindering learning effectiveness.

Purpose of the Study:

• To propose a novel Potential field Subgoal-based Multi-Agent reinforcement learning (PSMA) method.
• To unify the learning objectives of subgoal generation and subgoal reached stages in MARL.

Main Methods:

• Introduced potential fields (PF) to represent agent states and measure inter-agent interactions.
• Developed a state-to-PF representation model and a subgoal selector using an experience replay buffer.
• Defined an intrinsic reward function to guide agents toward subgoals while maximizing joint action-value.

Main Results:

• The proposed PSMA method effectively unifies two-stage learning objectives.
• PSMA demonstrated superior performance compared to state-of-the-art MARL methods.
• Achieved significant improvements on StarCraft II micro-management (SMAC) and Google Research Football (GRF) tasks in sparse reward settings.

Conclusions:

• The PSMA method offers a unified approach to subgoal learning in MARL.
• Potential fields provide an effective mechanism for representing agent states and interactions.
• PSMA significantly enhances MARL performance in complex, sparse reward environments.