FMRQ-A Multiagent Reinforcement Learning Algorithm for Fully Cooperative Tasks.

Area of Science:

• Artificial Intelligence
• Multiagent Systems
• Reinforcement Learning

Background:

• Multiagent systems often face challenges in coordinating agents for fully cooperative tasks.
• Existing reinforcement learning algorithms may struggle to converge to optimal solutions in complex cooperative environments.
• The need for efficient coordination mechanisms that do not require full observability of other agents' actions is critical.

Purpose of the Study:

• To propose a novel multiagent reinforcement learning algorithm, frequency of the maximum reward Q-learning (FMRQ), for fully cooperative tasks.
• To enable agents to converge to optimal Nash equilibria by utilizing a modified reinforcement signal.
• To demonstrate the algorithm's effectiveness in various cooperative scenarios.

Main Methods:

• Developed the frequency of the maximum reward Q-learning (FMRQ) algorithm.
• Modified the reinforcement signal to be the frequency of obtaining the highest global immediate reward.
• Agents share only their state and reward, not actions, at each step.
• Validated FMRQ through case studies including repeated games (two-player, three-player) and complex tasks (box-pushing, distributed sensor networks).

Main Results:

• FMRQ demonstrated convergence to optimal Nash equilibria in tested repeated game scenarios.
• Experimental results on box-pushing and distributed sensor network problems showed FMRQ's superior performance compared to other algorithms.
• The algorithm effectively optimizes the performance index in multiagent systems under cooperative settings.

Conclusions:

• FMRQ is an effective multiagent reinforcement learning algorithm for fully cooperative tasks.
• The use of reward frequency as a reinforcement signal facilitates convergence to optimal Nash equilibria.
• FMRQ offers improved performance and coordination capabilities in complex multiagent environments.