Deciding when to intervene: a Markov decision process approach

Area of Science:

• Decision Analysis
• Medical Informatics
• Operations Research

Background:

• Static decision-making models struggle with dynamic medical problems.
• Classical approaches like decision trees yield suboptimal solutions for time-sensitive interventions.

Purpose of the Study:

• To highlight the limitations of static decision models in dynamic scenarios.
• To introduce and apply a dynamic formalism using Markov decision processes (MPPs) for optimal intervention timing.
• To compare dynamic vs. static approaches in a medical context.

Main Methods:

• Developed a dynamic formalism based on Markov decision processes (MPPs).
• Applied the MPP approach to prophylactic surgery in mild hereditary spherocytosis.
• Utilized DT-Planner, a specialized graphical decision aid for dynamic processes.
• Compared the dynamic MPP policy against a static policy for the same medical problem.

Main Results:

• Static approaches were found to be inadequate for dynamic medical decision-making.
• The dynamic MPP approach provided a superior policy compared to the static approach.
• Optimal intervention timing was achieved by delaying surgery for certain patient groups.
• Significant gains were observed using the dynamic approach.

Conclusions:

• Dynamic decision processes, modeled via MPPs, are essential for optimizing medical interventions.
• Markov decision processes offer a more effective framework than static methods for complex medical decisions.
• The proposed dynamic approach leads to improved clinical outcomes by optimizing intervention timing.