Quantifying Fidelity and Utility in Synthetic Healthcare Data

Area of Science:

• Healthcare Machine Learning
• Data Privacy
• Artificial Intelligence

Background:

• High-quality clinical datasets are scarce, hindering predictive model development in healthcare.
• Strict privacy regulations pose significant challenges for accessing and utilizing sensitive patient data.
• Generative models offer a potential solution by creating synthetic data (SD) that preserves real data (RD) characteristics while protecting privacy.

Purpose of the Study:

• To introduce a structured framework for evaluating the fidelity and utility of synthetic data (SD) generated by tabular models.
• To assess the performance of three generative models: Conditional Tabular GAN (CTGAN), CopulaGAN, and Tabular Variational Autoencoder (TVAE).
• To determine the effectiveness of Pairwise Correlation Distance (PCD) and Wasserstein Distance (WSD) as metrics for SD quality.

Main Methods:

• Utilized the Pima Indians Diabetes Dataset to generate synthetic data (SD) using CTGAN, CopulaGAN, and TVAE.
• Quantified synthetic data fidelity using Pairwise Correlation Distance (PCD) and Wasserstein Distance (WSD).
• Evaluated synthetic data utility by measuring binary classification performance and comparing it to models trained on real data (RD).

Main Results:

• Optimized generative models successfully produced high-fidelity synthetic data (SD).
• The predictive accuracy of models trained on SD was comparable to those trained on real data (RD).
• Pairwise Correlation Distance (PCD) and Wasserstein Distance (WSD) proved to be reliable indicators of SD quality.

Conclusions:

• Generative models can effectively create high-fidelity synthetic data (SD) for healthcare machine learning applications.
• Optimized generative models mitigate privacy risks associated with real clinical data (RD) while maintaining predictive utility.
• Pairwise Correlation Distance (PCD) and Wasserstein Distance (WSD) are valuable metrics for optimizing generative models in healthcare.