Efficient multi-agent policy adaptation with Bayesian policy reuse and view-invariant contrastive awareness

Area of Science:

• Artificial Intelligence
• Multi-Agent Systems
• Reinforcement Learning

Background:

• Bayesian Policy Reuse (BPR) enables adaptation to non-stationary opponents but is limited to single-agent, fully observable settings.
• Partial observability and multi-agent coordination present significant challenges for existing policy adaptation frameworks.

Purpose of the Study:

• To propose BPR-VCA, integrating BPR with multi-view contrastive learning for efficient policy adaptation in partially observable multi-agent environments.
• To enable agents to learn from local observations and achieve consensus on global dynamics despite limited information.

Main Methods:

• BPR-VCA utilizes local observation trajectories for belief updates, addressing partial observability.
• A View-invariant Contrastive Awareness (VCA) module fosters consensus on environmental changes by integrating global and local views.
• Decentralized online belief updating allows agents to select appropriate joint policies.

Main Results:

• BPR-VCA demonstrated superior performance across four competitive scenarios.
• The framework achieved higher episodic and accumulated rewards compared to state-of-the-art baselines.
• Faster and more accurate opponent recognition, along with higher win rates, were observed.

Conclusions:

• BPR-VCA effectively addresses partial observability and multi-agent coordination challenges in policy adaptation.
• The proposed method offers a robust solution for real-world applications requiring adaptive AI agents in dynamic environments.