Utility-based Analysis of Statistical Approaches and Deep Learning Models for Synthetic Data Generation With Focus on...

Area of Science:

• Medical Data Science
• Machine Learning in Healthcare
• Synthetic Data Generation

Background:

• Advancements in Generative Adversarial Networks and LLMs enable medical data synthesis.
• Deep learning methods offer potential for high-quality, realistic datasets, crucial for healthcare ML.
• Challenges persist in accurately capturing complex associations within medical datasets.

Purpose of the Study:

• Evaluate various Synthetic Data Generation (SDG) methods for replicating medical dataset correlation structures.
• Assess SDG method performance on downstream tasks using Random Forests and other models.
• Compare statistical (synthpop, copula) and deep learning (ctgan, tvae, LLMs) SDG approaches.

Main Methods:

• Evaluated SDG methods on simulated and real-world medical datasets (body performance, breast cancer, diabetes).
• Assessed data quality using correlation matrices, propensity score MSE (pMSE), and F1-scores.
• Compared synthetic data utility by training models on synthetic data and testing on real data.

Main Results:

• Statistical methods (synthpop, copula) consistently outperformed deep learning approaches in preserving correlation structures.
• Synthpop was the most effective SDG method overall.
• Deep learning methods, including LLMs, showed variable performance, struggling with numerical dependencies and smaller datasets.

Conclusions:

• Statistical methods, especially synthpop, are superior for synthetic tabular data generation, offering robustness and utility.
• Copula methods show promise but have limitations with integer variables.
• Deep learning methods underperform for general tabular synthetic data but may have niche applications.