Markov decision process design: A framework for integrating strategic and operational decisions.

Area of Science:

• Operations Research
• Systems Engineering
• Decision Science

Background:

• Designing systems for repeated use often involves balancing upfront design costs with future operational expenses.
• Uncertainty in operational conditions complicates optimal system design, making traditional approaches insufficient.

Purpose of the Study:

• To develop a unified modeling framework for the optimal design of systems intended for repeated use under uncertainty.
• To simultaneously minimize initial design costs and expected future operational costs.

Main Methods:

• The study integrates a mixed-integer program for the design phase and discounted-cost infinite-horizon Markov decision processes for the operational phase.
• A bilevel mixed-integer linear programming formulation was derived to address the integrated problem.
• Computational studies were conducted on realistic problem instances.

Main Results:

• The proposed framework successfully integrates system design and operational planning under uncertainty.
• The bilevel formulation provides a method for simultaneously optimizing design and operational costs.
• Numerical results demonstrate the solvability of realistic instances.

Conclusions:

• The developed modeling framework offers an effective approach to optimally design systems for repeated use in uncertain environments.
• This research provides a valuable tool for decision-makers aiming to minimize total system costs over its lifecycle.