Related Experiment Videos
Leveraging routine scheduling patterns to unshift Date-Shifted clinical data
Andrzej Kudlicki1, Bradley A Malin2, Elmer V Bernstam3
1D. Bradley McWilliams School for Biomedical Informatics, The University of Texas Health Science Center in Houston, United States.
Objective:
Date-shifting is a common strategy for preserving temporal relationships in de-identified clinical data, premised on the assumption that all candidate shifts in a chosen range are equally likely to a recipient. We assess whether routine scheduling patterns - weekday concentration of outpatient activity and predictable institutional holiday closures - invalidate this assumption, and quantify the resulting risk of date recovery as a function of dataset characteristics.
Methods:
We formalize the unshifting problem as Bayesian inference over candidate shifts, scoring each candidate by the likelihood that reconstructed dates are consistent with weekday and holiday scheduling patterns. Model parameters were estimated empirically from 95.9 million encounters across 4.1 million patient records in one large academic health system over 5 years. We then simulated unshifting on 1,617,988 outpatients (with 2 + distinct event dates) across 22 shift ranges from 7 days to 5 years, comparing the recipient's argmax against the known true shift.
Results:
Ambulatory visits occurred on weekdays 98.0% of the time, with some encounter subtypes at 100%; a similar pattern affects non-weekend holidays (at least six or seven institutional closure days per year, depending on which fixed holidays fall on weekdays). A recipient targeting only patients where posterior confidence reaches τ ≥ 0.95 correctly recovered the true shift for 5.9% of the population under a 30-day shift, with 88.4% precision. Even under a one-year shift, patients with over 100 distinct encounter dates faced a 34% chance of correct date unshifting.
Conclusion:
The unshifting vulnerability is substantial and depends predictably on encounter density, shift range, and the institutional holiday calendar. It affects datasets of all sizes, including case reports with relative timelines. The same Bayesian framework that enables unshifting also serves as a pre-release risk assessment tool: data custodians can compute per-patient vulnerability scores and apply targeted mitigations before releasing date-shifted data.