MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Trees

Area of Science:

• Artificial Intelligence
• Machine Learning
• Multi-Agent Systems

Background:

• Existing multi-agent reinforcement learning (MARL) systems often use black-box neural networks, hindering understanding of decision-making processes.
• Interpretable MARL approaches typically exhibit limited expressivity and performance.
• A gap exists between the need for interpretable and high-performing MARL systems.

Purpose of the Study:

• To develop a novel interpretable architecture for MARL that balances performance with explainability.
• To enable explicit representation of decision processes and agent contributions in MARL.
• To provide insights into the cooperation mechanisms within MARL systems.

Main Methods:

• Proposed MIXing Recurrent soft decision Trees (MIXRTs), a novel architecture combining recurrent structures with soft decision trees.
• Utilized a value decomposition framework to linearly assign credit to individual agents based on local observations.
• Incorporated theoretical analysis to ensure additivity and monotonicity in joint action value factorization.

Main Results:

• MIXRTs provide explicit decision process representation through root-to-leaf paths.
• The architecture effectively reflects individual agent contributions to team objectives.
• Evaluations on complex tasks (Spread, StarCraft II) show MIXRTs achieve competitive performance while offering clear explanations.

Conclusions:

• MIXRTs successfully bridge the gap between interpretability and performance in MARL.
• The proposed method offers a new way to understand and interpret MARL cooperation mechanisms.
• MIXRTs pave the way for more transparent and effective MARL systems.