Learning Roles With Emergent Social Value Orientations.

Area of Science:

• Artificial Intelligence
• Multi-Agent Systems
• Social Science

Background:

• Social dilemmas arise when individual rational choices lead to collectively irrational outcomes.
• Multi-agent reinforcement learning (MARL) utilizes social science concepts like social value orientations (SVOs) to address these challenges in cooperative tasks.

Purpose of the Study:

• To propose a novel learning framework, Learning Roles with Emergent SVOs (RESVO), for intertemporal social dilemmas (ISDs).
• To transform role learning into the emergence of SVOs, promoting cooperation through altruistic reward sharing.

Main Methods:

• Introduced the "division of labor" mechanism from human society into MARL.
• Developed the RESVO framework, conditioning policies on roles within an SVO-based embedding space.
• Utilized a novel rank regularizer and mutual information maximizer for policy conditioning.

Main Results:

• RESVO successfully enabled agents to learn roles and develop SVOs.
• Demonstrated stable division of labor and cooperation in ISDs of varying complexity.
• Showcased the effectiveness of altruism in reward sharing for cooperative task completion.

Conclusions:

• RESVO provides a promising solution for intertemporal social dilemmas by integrating role emergence with SVOs.
• The framework facilitates stable cooperation and division of labor in multi-agent systems.
• This approach highlights the potential of social science principles in advancing MARL research.