Privacy-Preserving Model Transcription With Differentially Private Synthetic Distillation

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Privacy

Background:

• Deep learning models trained on private data risk privacy leakage.
• Existing deployment methods may expose sensitive information.
• Need for secure model deployment solutions is critical.

Purpose of the Study:

• To develop a data-free, model-to-model conversion technique for privacy-preserving AI deployment.
• To introduce a novel approach for transcribing models while guaranteeing privacy.
• To enable the generation of private synthetic data for downstream tasks.

Main Methods:

• Proposed differentially private synthetic distillation, a cooperative-competitive learning framework.
• Utilized a trainable generator to create synthetic data without accessing private datasets.
• Implemented a three-player optimization: generator, teacher model, and student model.
• Employed flexible data or label noisy perturbation for differential privacy.
• Applied adversarial training where the student model acts as a discriminator.

Main Results:

• The proposed approach guarantees differential privacy and convergence.
• The transcribed student model demonstrates strong performance and robust privacy protection.
• The generator successfully produces private synthetic data suitable for downstream tasks.
• Outperformed 26 state-of-the-art methods in extensive experiments.

Conclusions:

• Differentially private synthetic distillation offers an effective data-free solution for privacy-preserving model deployment.
• The method ensures model utility and data confidentiality.
• The generated synthetic data can be valuable for future machine learning tasks.