Evaluating Identity Disclosure Risk in Fully Synthetic Health Data: Model Development and Validation

Area of Science:

• Data privacy
• Statistical modeling
• Health informatics

Background:

• Growing interest in data synthesis for secondary analysis.
• Need for comprehensive privacy risk models for fully synthetic data.
• Overfitting generative models can lead to re-identification risks.

Purpose of the Study:

• Develop and apply a methodology for evaluating identity disclosure risks in fully synthetic data.
• Introduce a "meaningful identity disclosure risk" model.
• Assess privacy risks associated with synthetic health and social science data.

Main Methods:

• Presented a comprehensive risk model evaluating identity disclosure and adversary's ability to learn new information.
• Applied the model to synthetic samples from Washington State Hospital discharge and Canadian COVID-19 datasets.
• Utilized a sequential decision tree process for data synthesis.

Main Results:

• Meaningful identity disclosure risk was below the 0.09 threshold for both synthesized samples (0.0198 and 0.0086).
• Risks were 4-5 times lower than those of the original datasets.
• Demonstrated considerable reduction in meaningful identity disclosure risks through synthesis.

Conclusions:

• A comprehensive identity disclosure risk model for fully synthetic data has been presented.
• Data synthesis can substantially reduce meaningful identity disclosure risks.
• The developed risk model can be used for future privacy evaluations of synthetic data.