Weighted Q-learning for optimal dynamic treatment regimes with nonignorable missing covariates

Area of Science:

• Biostatistics
• Medical Informatics
• Clinical Decision Support

Background:

• Dynamic treatment regimes (DTRs) guide sequential medical decisions.
• Electronic medical record (EMR) data present challenges due to nonignorable missing covariates.
• Standard Q-learning struggles with missing covariates in DTR estimation.

Purpose of the Study:

• To develop novel methods for estimating optimal DTRs with nonignorable missing covariates.
• To address the issue of missing pseudo-outcomes in backward induction algorithms.
• To investigate optimal fluid strategies for sepsis patients using EMR data.

Main Methods:

• Proposed two weighted Q-learning approaches using inverse probability weighting.
• Employed estimating equations with nonresponse instrumental variables or sensitivity analysis.
• Derived asymptotic properties and conducted extensive simulation studies.

Main Results:

• The proposed weighted Q-learning methods demonstrate improved performance in simulations.
• Evaluated and compared the finite-sample performance against alternative methods.
• Applied the methods to real-world EMR data from the MIMIC database.

Conclusions:

• Weighted Q-learning offers a robust solution for DTR estimation with nonignorable missing data.
• The findings provide a more accurate approach to optimizing sequential treatment decisions.
• Identified potential optimal fluid strategies for intensive care unit sepsis patients.