Guiding Multiagent Multitask Reinforcement Learning by a Hierarchical Framework With Logical Reward Shaping.

Area of Science:

• Artificial Intelligence
• Multiagent Systems
• Reinforcement Learning

Background:

• Multiagent hierarchical reinforcement learning (MAHRL) effectively addresses complex decision-making but is often limited to single tasks due to traditional reward functions.
• Existing MAHRL algorithms struggle with multitasks, necessitating a more flexible reward system for broader applicability.

Purpose of the Study:

• To design a novel multiagent cooperative algorithm using logic reward shaping (LRS) for effective multitasks completion.
• To enhance the interpretability and credibility of agent decisions through logic-based reward structures.
• To improve coordination and cooperation among multiple agents in complex environments.

Main Methods:

• Logic reward shaping (LRS) utilizes linear-time temporal logic (LTL) to define subtask relationships and guide reward structures.
• A value iteration technique evaluates individual agent actions to shape a coordination reward function.
• Agents learn through experiential learning, assessing their status and completing subtasks based on LTL specifications.

Main Results:

• The proposed LRS algorithm demonstrated improved performance in multiagent multitasks scenarios.
• Experiments in Minecraft World and Office World validated the algorithm's effectiveness.
• The approach enhanced agent coordination and the ability to complete complex, multi-step tasks.

Conclusions:

• Logic reward shaping provides a flexible and interpretable method for multiagent reinforcement learning in multitasks settings.
• The developed algorithm successfully enhances cooperation and decision-making capabilities in complex environments.
• This research offers a promising direction for advancing MAHRL in real-world applications.