Improving Clinical Validity in Synthetic Electronic Health Record Generation Using Best-of-N Sampling: Comparative...

Synthetic electronic health record generation is limited not only by statistical fidelity but also by clinical validity. Records that appear statistically plausible may still violate hard structural, physiological, or relational constraints.

This study evaluated best-of-N constraint-minimizing selection as an inference-time strategy for improving clinical validity and characterized when such selection succeeds or fails as a function of the generator's valid support mass (pvalid).

We evaluated Wasserstein generative adversarial network with gradient penalty (WGAN-GP) and conditional tabular generative adversarial network (CTGAN) on 3 public clinical tabular datasets: stroke prediction (n=5110), diabetes health indicators (n=100,000), and cardiovascular disease (n=68,599). For each generator, we compared random sampling, naive clipping, and best-of-N selection (N ∈ {8, 16, 128}). Validity was assessed via dataset-specific rule violations; fidelity via Kolmogorov-Smirnov (KS) statistics and correlation preservation; utility via TSTR (train-on-synthetic-test-on-real) AUC (area under the receiver operating characteristic curve); and privacy via membership inference attack (MIA) AUC and distance to closest record (DCR).

Best-of-N effectiveness was governed by the base generator's valid support mass. In stroke, WGAN-GP had pvalid=0, and all selection strategies, including strict rejection across 5000 draws, produced 500/500 (100%) invalid samples. In cardiovascular data, WGAN-GP had pvalid=0.12; best-of-16 reduced violations from 441/500 (88.2%) to 65/500 (13.0%), and best-of-128 eliminated them entirely (0/500), matching theoretical predictions exactly. In diabetes, WGAN-GP random sampling was already fully valid (0/500 violations), and best-of-N primarily shifted utility. Across all 3 datasets, CTGAN+best-of-16 achieved 0/500 violations with strong fidelity (mean KS≤0.12) and utility (TSTR AUC up to 0.81). MIA AUC remained near-random guessing (approximately 0.50) throughout, though DCR-based proximity concentration increased markedly under CTGAN selection (reaching 22.6% on stroke and 100% on cardiovascular data).

Best-of-N functions as a bounded-budget feasibility filter rather than a universal repair mechanism. Its effectiveness depends critically on the generator's valid support mass. CTGAN+best-of-16 offered the strongest overall trade-off across clinical validity, fidelity, utility, and privacy in our experiments.