Deep Learning With Data Privacy via Residual Perturbation.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Privacy

Background:

• Deep learning (DL) models require robust data privacy measures.
• Existing privacy-preserving DL methods often compromise utility and increase computational costs.
• Differential privacy (DP) is a key standard for privacy guarantees.

Purpose of the Study:

• To propose a novel, computationally efficient method for privacy-preserving deep learning.
• To theoretically and empirically validate the proposed method's effectiveness in maintaining data privacy and model utility.
• To compare the proposed method against state-of-the-art differentially private approaches.

Main Methods:

• A novel stochastic differential equation-based residual perturbation technique is introduced.
• Gaussian noise is injected into residual mappings of ResNets for privacy preservation.
• Theoretical analysis to prove differential privacy (DP) and reduced generalization gap.
• Empirical evaluation comparing against differentially private stochastic gradient descent (DPSGD).

Main Results:

• The proposed residual perturbation method guarantees differential privacy (DP).
• The method is shown to reduce the generalization gap in deep learning models.
• Residual perturbation demonstrates computational efficiency.
• Empirically superior utility maintenance compared to state-of-the-art DPSGD without sacrificing membership privacy.

Conclusions:

• Stochastic differential equation-based residual perturbation offers an effective solution for privacy-preserving deep learning.
• This method balances strong privacy guarantees with high model utility and computational efficiency.
• It represents a significant advancement over existing differentially private training methods like DPSGD.