Q-learning via deep learning-based Buckley-James method for non-linear censored data

Area of Science:

• Biostatistics
• Machine Learning in Healthcare
• Survival Analysis

Background:

• Personalized treatment is crucial for better patient outcomes, particularly with right-censored survival data.
• Existing methods struggle with nonlinearities and censoring in complex clinical data.

Purpose of the Study:

• To introduce Dynamic Deep Buckley-James Q-Learning, a novel algorithm for estimating optimal dynamic treatment regimes.
• To address challenges of right-censoring and nonlinear modeling in survival data analysis.

Main Methods:

• Integrating deep learning with the Buckley-James method for counterfactual Q-learning.
• Estimating potential survival outcomes under hypothetical treatment sequences using a counterfactual framework.
• Maximizing expected imputed survival reward under counterfactual scenarios.

Main Results:

• The algorithm robustly estimates optimal dynamic treatment regimes by capturing nonlinear covariate-treatment interactions.
• Unbiased Q-function estimation is achieved despite time-dependent covariates and right censoring.
• Demonstrated superior performance in predictive accuracy and treatment decision-making.

Conclusions:

• Dynamic Deep Buckley-James Q-Learning offers a powerful framework for individualized care in complex clinical settings.
• The method significantly improves patient outcomes through enhanced personalized treatment strategies.
• Validates superior performance through simulation studies and real-world data analysis.