Improving Clinical Validity in Synthetic Electronic Health Record Generation Using Best-of-N Sampling: Comparative...

Area of Science:

• Synthetic Data Generation
• Health Informatics
• Machine Learning

Background:

• Synthetic electronic health records (EHRs) face limitations in statistical fidelity and clinical validity.
• Plausible synthetic data can still violate critical physiological or relational constraints.

Purpose of the Study:

• Evaluate best-of-N constraint-minimizing selection for enhancing clinical validity of synthetic EHRs.
• Characterize the success and failure of this strategy based on generator valid support mass (pvalid).

Main Methods:

• Compared Wasserstein generative adversarial network with gradient penalty (WGAN-GP) and conditional tabular generative adversarial network (CTGAN) on three public clinical datasets.
• Assessed random sampling, naive clipping, and best-of-N selection (N={8, 16, 128}).
• Evaluated validity, fidelity (Kolmogorov-Smirnov statistics, correlation preservation), utility (TSTR AUC), and privacy (MIA AUC, DCR).

Main Results:

• Best-of-N effectiveness is tied to the generator's pvalid; WGAN-GP with pvalid=0 failed to produce valid samples.
• For WGAN-GP with pvalid=0.12, best-of-128 selection eliminated violations (0/500).
• CTGAN+best-of-16 achieved zero violations with strong fidelity and utility (TSTR AUC up to 0.81), maintaining near-random privacy.

Conclusions:

• Best-of-N acts as a feasibility filter, its efficacy dependent on the generator's valid support mass.
• CTGAN+best-of-16 demonstrated the optimal balance across clinical validity, fidelity, utility, and privacy in the evaluated scenarios.