Evolving Robust Policy Coverage Sets in Multi-Objective Markov Decision Processes Through Intrinsically Motivated...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Reinforcement Learning

Background:

• Real-world decision-making often involves multiple conflicting objectives, leading to multi-objective Markov decision processes (MOMDPs).
• Conventional single-objective reinforcement learning struggles with MOMDPs, especially when optimal compromises are unknown.
• Existing multi-objective reinforcement learning methods face challenges with computational complexity, time, and adaptability to dynamic environments.

Purpose of the Study:

• To develop an adaptive, online, and robust method for solving multi-objective reinforcement learning problems.
• To overcome the limitations of existing methods in terms of computational cost and adaptability to non-stationary environments.
• To find an optimal coverage set of non-dominated policies that can satisfy diverse user preferences.

Main Methods:

• Proposed a novel developmental method employing adversarial self-play.
• Integrated an intrinsically motivated preference exploration component with a policy coverage set optimization component.
• The method robustly evolves a convex coverage set of policies based on explored preferences.

Main Results:

• Demonstrated the effectiveness of the proposed method experimentally.
• Showcased superior performance compared to state-of-the-art multi-objective reinforcement learning methods.
• Validated the method's effectiveness in both stationary and non-stationary environments.

Conclusions:

• The proposed developmental method offers a robust and adaptive solution for multi-objective reinforcement learning.
• Adversarial self-play and intrinsic motivation enhance policy coverage set evolution.
• The method provides a promising alternative for complex decision-making problems with conflicting objectives.