Regression discontinuity in Time: Evaluating the impact of evolving digital health interventions

Clinics continue to adopt digital health interventions (DHIs) in which algorithms analyze data to help direct patient care. Changes to these algorithms are rarely evaluated rigorously, despite their potential to affect patient outcomes. The regression discontinuity in time (RDT) design may be used to estimate the causal effect of such changes but has received little attention in medical literature.

We conducted a retrospective study of continuous glucose monitor (CGM) data from youth with type 1 diabetes enrolled in the 4T Program from November 2020 to March 2022. In 2020, the clinic used an algorithm that directed patients for provider review based on the number of glucose targets not being met (e.g., time in range (TIR) < 65 %). In September 2021, the clinic adopted a new algorithm that prioritized the review of patients experiencing level 2 hypoglycemia (glucose < 54 mg/dl). We evaluated the validity of the RDT framework in this setting and estimated the impact of this change in directed care on weekly TIR.

There were 247 patients with 11,297 weekly TIR observations. Robustness checks supported the validity of the RDT design. Patients with level 2 hypoglycemia were prioritized for review, with no significant change to population-level TIR (-0.2% points; 95% CI:[-4.8, 3.5]).

We demonstrate the feasibility of using the RDT framework to estimate the impact of algorithmic changes to a DHI on patient care. As algorithm-directed care increases in clinical practice, this approach can serve as a diagnostic tool to measure how operational changes impact outcomes.