Estimating optimal dynamic treatment regimes with Gaussian process emulation

Identifying dynamic treatment regimes (DTRs) is a key objective in precision medicine. Value search approaches, including (Bayesian) dynamic marginal structural models offer an attractive approach to estimation by mapping candidate regimes to their expected outcome. As parametric models for the expected outcomes may be mis-specified and lead to incorrect conclusions, a grid search over candidate DTRs has been proposed, but this may be computationally prohibitive and also subject to high uncertainty in the estimated value function. These inferential challenges can be addressed using Gaussian process ($\mathcal {GP}$) optimization methods with estimators for the causal effect of adherence to a specified DTR. We demonstrate how to identify optimal DTRs using this approach in a variety of settings, including when the value function is multi-modal and show that the $\mathcal {GP}$ modeling approach that recognizes noise in the estimated response surface leads to improved results as compared to a grid search approach. Further, we show that a grid search may not yield a robust solution and that it often utilizes information less efficiently than a $\mathcal {GP}$ approach. The proposed approach is used to understand tailoring of HIV therapy to optimize CD4 cell counts.