Understanding Deep Gradient Leakage via Inversion Influence Functions.

Area of Science:

• Artificial Intelligence
• Machine Learning Security
• Data Privacy

Background:

• Deep Gradient Leakage (DGL) poses significant privacy risks in distributed learning by recovering training data from shared gradients.
• Understanding the mechanisms of privacy leakage in deep networks is crucial for developing effective defenses but remains challenging due to their black-box nature.

Purpose of the Study:

• To develop a novel method, the Inversion Influence Function (I²F), for analyzing privacy leakage in deep learning.
• To establish a scalable and efficient tool for understanding when and how privacy leakage occurs during gradient-based attacks.

Main Methods:

• Propose the Inversion Influence Function (I²F), which implicitly solves the Deep Gradient Leakage problem.
• I²F requires only oracle access to gradients and Jacobian-vector products, making it scalable for deep networks.
• Empirically validate I²F's approximation of DGL across diverse model architectures, datasets, and defenses.

Main Results:

• I²F effectively approximates Deep Gradient Leakage across various settings, demonstrating its generalizability.
• The study provides insights into optimal gradient perturbation strategies for privacy protection.
• Analysis reveals inequities in privacy protection and suggests methods for privacy-preferred model initialization.

Conclusions:

• The Inversion Influence Function (I²F) is a scalable and effective tool for analyzing privacy leakage in deep learning.
• I²F enables a deeper understanding of Deep Gradient Leakage, facilitating the development of more robust privacy-preserving techniques.
• This work contributes to enhancing data privacy in distributed machine learning systems.