A comprehensive evaluation framework for synthetic medical tabular data generation.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Data Privacy

Background:

• Machine learning (ML) drives healthcare advancements but requires large datasets, often hindered by patient privacy concerns.
• Synthetic data generation is a promising solution for accessing large-scale training data while safeguarding patient confidentiality.

Purpose of the Study:

• To introduce a comprehensive evaluation framework for synthetic tabular medical data.
• To assess synthetic data across quality, privacy, usability, and computational complexity.
• To ensure synthetic data utility for ML applications without compromising patient privacy.

Main Methods:

• Developed a novel evaluation framework for synthetic medical data.
• Applied six state-of-the-art generative models to create synthetic Electronic Health Record (EHR) datasets.
• Evaluated synthetic data using the framework, focusing on quality, privacy, usability, and computational aspects.

Main Results:

• The framework identified critical shortcomings in synthetic data, including amplified duplicate rows and out-of-range values.
• Traditional statistical similarity measures overlooked these critical issues.
• The evaluation incorporated outlier detection, privacy risks, and domain-specific constraints for a broader assessment.

Conclusions:

• The proposed framework offers a more robust assessment of synthetic medical data than conventional methods.
• It is crucial for identifying subtle data generation flaws that impact ML model reliability and patient privacy.
• The framework ensures synthetic data is suitable for ML while maintaining data confidentiality.