Multi-Objective Markov Decision Processes for Data-Driven Decision Support.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Operations Research

Background:

• Sequential decision-making systems often struggle to accommodate diverse and evolving user preferences.
• Existing methods may not efficiently handle multiple objectives or provide a comprehensive set of optimal policies.

Purpose of the Study:

• To introduce a novel methodology for developing sequential decision support systems using Multi-Objective Markov Decision Processes (MOMDPs).
• To enable systems that cater to multiple decision-makers with potentially time-varying preferences.
• To compute policies for all relevant preference functions simultaneously from data.

Main Methods:

• Extension of the fitted-Q iteration algorithm to handle multiple objectives.
• Simultaneous computation of policies for all scalarization functions (preference functions).
• Development of a new solution concept for scenarios with similar expected outcomes across different actions.

Main Results:

• The methodology successfully computes policies for a range of preferences from continuous-state, finite-horizon data.
• Addressed conceptual and computational challenges inherent in multi-objective reinforcement learning.
• Demonstrated applicability using real-world data from the Clinical Antipsychotic Trials of Intervention Effectiveness.

Conclusions:

• The proposed MOMDP-based approach enhances decision support systems by accommodating diverse user preferences.
• It provides decision-makers with a broader selection of optimal policies, increasing utility and choice.
• This method offers a flexible and data-driven framework for personalized sequential decision-making.