A Multi-Agent Continual reinforcement learning framework with multi-Timescale replay and dynamic task classification.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Multi-Agent Systems

Background:

• Traditional reinforcement learning struggles with catastrophic forgetting and poor knowledge transfer in non-stationary, multi-agent settings.
• Dynamic environments pose significant challenges for sequential task learning in multi-agent systems.

Purpose of the Study:

• To propose an innovative Multi-Agent Continual Reinforcement Learning (MACRL) framework.
• To address catastrophic forgetting and enhance cross-task knowledge transfer in dynamic multi-agent systems.
• To enable scalable collaborative decision-making in complex, evolving environments.

Main Methods:

• Introduced a Multi-Timescale Replay (MTR) buffer for hierarchical experience storage across timescales.
• Developed a dynamic task classification mechanism using an attention-based contextual encoder to measure task similarity.
• Implemented adaptive policy routing to minimize inter-task interference.

Main Results:

• The MACRL framework achieved higher average returns in sequential task learning on cooperative benchmarks (LBF and PP) compared to baselines.
• Demonstrated superior zero-shot generalization performance.
• Ablation studies confirmed the effectiveness of MTR and task classification in mitigating catastrophic forgetting.

Conclusions:

• The proposed MACRL framework offers a scalable solution for continual learning in dynamic multi-agent systems.
• The MTR buffer and dynamic task classification are crucial for retaining knowledge and minimizing interference.
• This approach improves collaborative decision-making in complex, evolving environments.