Social satisficing: Multi-agent reinforcement learning with satisficing agents.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Reinforcement Learning

Background:

• Reinforcement learning agents require limited exploration for efficient trial-and-error learning.
• Limiting exploration can lead to suboptimal solutions due to local, bottom-up learning.
• Risk-sensitive satisficing (RS) value functions offer a top-down approach but require an appropriate aspiration level.

Purpose of the Study:

• To address the open problem of determining the aspiration level in reinforcement learning.
• To propose social satisficing, a novel framework for multi-agent reinforcement learning.
• To enhance learning efficiency and optimality in agents.

Main Methods:

• Developed a social satisficing framework for multi-agent reinforcement learning.
• Agents determine aspiration levels through information sharing.
• Converted episodic aspiration levels into local, state-wise aspiration levels.
• Conducted simulations in a challenging environment (SuboptimaWorld) with numerous suboptimal goals.

Main Results:

• The proposed social satisficing method demonstrated higher learning efficiency compared to existing methods.
• The framework effectively prevented agents from converging to suboptimal solutions.
• The method showed an ability to autonomously adjust the exploration scope.
• Minimal shared information was required for effective learning.

Conclusions:

• Social satisficing provides an effective approach for multi-agent reinforcement learning.
• The framework enhances learning efficiency and solution optimality.
• This study offers insights into social behaviors relevant to AI and machine learning.