Large Language Models for Synthetic Tabular Health Data: A Benchmark Study.

Area of Science:

• Health Informatics
• Artificial Intelligence in Medicine
• Data Science

Background:

• Synthetic tabular health data is vital for research due to privacy concerns and data scarcity.
• Large Language Models (LLMs) and Generative Adversarial Networks (GANs) are leading methods for synthetic data generation.

Purpose of the Study:

• To compare the performance of transformer-based LLMs against a reference GAN for synthetic tabular health data generation.
• To evaluate the impact of LLM model size and training dataset size on synthetic data utility.

Main Methods:

• Compared Pythia LLM models (14M to 1B parameters) with CTGAN (a reference GAN).
• Used generated synthetic data to train random forest classifiers for prediction tasks on real-world data.

Main Results:

• LLM performance improved with increasing model size, surpassing the GAN baseline.
• Smallest LLMs (14M parameters) demonstrated comparable performance to GANs.
• A positive correlation was observed between training dataset size and model performance.

Conclusions:

• LLMs are effective for generating high-utility synthetic tabular health data, with performance scaling with model size.
• LLM-based synthetic data generation offers a viable alternative to GANs for healthcare research.
• Further research is needed on real-world implementation challenges and considerations for LLM synthetic data.