Optimization of multi-stage dynamic treatment regimes utilizing accumulated data

Area of Science:

• Biostatistics
• Machine Learning in Healthcare
• Clinical Trial Design

Background:

• Medical therapies often involve sequential treatment decisions based on patient history.
• Dynamic treatment regimes optimize these sequences for improved patient outcomes.
• Standard Q-learning can accumulate bias when outcome models are misspecified.

Purpose of the Study:

• To introduce a modified Q-learning algorithm to reduce bias in dynamic treatment regimes.
• To develop a robust method for optimizing sequential medical treatments.
• To improve the identification of optimal treatment policies in complex healthcare settings.

Main Methods:

• A modified Q-learning algorithm employing backward induction for sequential optimization.
• Development of computational algorithms, estimators, and closed-form variance formulas.
• Simulation studies to evaluate performance with misspecified outcome models.

Main Results:

• The modified Q-learning method demonstrated a higher probability of identifying optimal treatment regimes.
• Reduced accumulated bias compared to standard Q-learning, especially with numerous treatment stages.
• Successful application to a prostate cancer study and comparison of two-stage treatment sequences.

Conclusions:

• The modified Q-learning offers a more reliable approach for optimizing dynamic treatment regimes.
• This method is particularly valuable in scenarios with potential outcome model misspecification.
• The findings have implications for personalized medicine and adaptive clinical trial designs.