Model learning and knowledge sharing for a multiagent system with Dyna-Q learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Multiagent Systems

Background:

• Multiagent systems require efficient environmental modeling for accelerated learning.
• Exploration of unvisited states is a bottleneck in complex environments.
• Knowledge sharing can enhance individual agent learning and global model accuracy.

Purpose of the Study:

• To develop a model-based reinforcement learning method for efficient multiagent environmental modeling.
• To reduce memory consumption and accelerate the learning process in complex multiagent systems.
• To facilitate effective knowledge sharing among agents.

Main Methods:

• A Dyna-Q architecture tailored for multiagent systems using a tree structure for modeling.
• Generating virtual experiences from a tree-model built from real experiences.
• Employing resampling techniques to graft partial tree branches for knowledge sharing, avoiding full tree merging.

Main Results:

• The proposed method achieves efficient modeling with reduced memory consumption.
• Virtual experience generation significantly reduces learning time.
• Knowledge sharing through partial tree grafting enhances modeling accuracy and provides valid simulated experiences.

Conclusions:

• The developed tree-based model-based reinforcement learning method accelerates learning in multiagent systems.
• The knowledge sharing mechanism improves sample efficiency and overall learning performance.
• The approach is effective for multiagent cooperation applications.