Deep Q-Managed: a new framework for multi-objective deep reinforcement learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Reinforcement Learning

Background:

• Multi-objective optimization presents challenges like the curse of dimensionality and overestimation bias.
• Existing algorithms struggle to efficiently identify the entire Pareto Front in complex scenarios.

Purpose of the Study:

• Introduce Deep Q-Managed, a novel multi-objective reinforcement learning (MORL) algorithm.
• Enable the discovery of all policies within the Pareto Front.
• Enhance multi-objective optimization using deep learning techniques.

Main Methods:

• Integrate deep learning techniques, specifically Double and Dueling Networks, into the MORL framework.
• Mitigate dimensionality and overestimation bias through advanced network architectures.
• Apply the algorithm to deterministic episodic environments.

Main Results:

• Deep Q-Managed successfully attains non-dominated multi-objective policies across various Pareto Front complexities (convex, concave, mixed).
• Consistent achievement of maximum hypervolume values on standard MORL benchmarks (DST, BST, MBST).
• Demonstrated ability to locate all Pareto Front points effectively.

Conclusions:

• Deep Q-Managed is a proficient algorithm for discovering comprehensive Pareto Fronts in deterministic settings.
• The algorithm shows robustness and versatility for applications in robotics, finance, and healthcare.
• Future work will focus on extending the algorithm to stochastic environments.