Deep Learning with Gaussian Differential Privacy.

Area of Science:

• Computer Science
• Machine Learning
• Privacy-Preserving Technologies

Background:

• Deep learning models require training on sensitive datasets, necessitating robust privacy constraints.
• Existing privacy definitions like differential privacy have limitations in handling composition and subsampling for neural network training.
• Refined privacy analysis is crucial for secure and accurate deep learning applications.

Purpose of the Study:

• To analyze the privacy guarantees of deep neural network training using the f-differential privacy framework.
• To derive analytically tractable expressions for privacy guarantees of stochastic gradient descent and Adam.
• To demonstrate improvements in privacy analysis and prediction accuracy compared to prior methods.

Main Methods:

• Utilized the f-differential privacy definition for a refined privacy analysis.
• Derived analytical expressions for privacy guarantees of stochastic gradient descent and Adam.
• Conducted experiments on image classification, text classification, and recommender systems.

Main Results:

• Developed a new privacy analysis framework based on f-differential privacy.
• Achieved improved privacy guarantees and analytically tractable expressions for common optimization algorithms.
• Demonstrated enhanced prediction accuracy without compromising privacy budgets through parameter tuning.
• Experimental validation across diverse machine learning tasks confirmed theoretical improvements.

Conclusions:

• The f-differential privacy framework offers a more effective approach to privacy analysis in deep learning.
• This framework enables better trade-offs between prediction accuracy and privacy guarantees.
• The findings suggest practical implications for developing more secure and accurate deep learning models.