Privacy Preserving Decentralized Learning With Positive-Incentive Noise

Area of Science:

• Computer Science
• Machine Learning
• Cybersecurity

Background:

• Decentralized learning faces privacy challenges due to local data sensitivity.
• The privacy-utility tradeoff hinders the effectiveness of privacy-preserving algorithms.
• Colluding inference attacks pose a significant threat to decentralized systems.

Purpose of the Study:

• To develop a novel mechanism (PING) that mitigates the negative impact of privacy noise on convergence in decentralized learning.
• To propose a privacy-preserving algorithm (PP-DPIN) that defends against sophisticated inference attacks.
• To provide robust privacy quantification and analyze convergence rates for decentralized learning.

Main Methods:

• Introduction of the Positive-Incentive Noise Generator (PING) utilizing network topologies and encryption.
• Development of the PP-DPIN algorithm integrating differential privacy and differential information entropy.
• Establishment of convergence rates under stochastic convex and nonconvex settings.

Main Results:

• PING generates correlated noise, preserving convergence while defending against attacks.
• PP-DPIN offers strong privacy guarantees for at least half of the nodes.
• Demonstrated linear speedup relative to network size and superior performance in computer vision tasks.

Conclusions:

• PING and PP-DPIN effectively address the privacy-utility tradeoff in decentralized learning.
• The proposed methods provide robust privacy guarantees and improved convergence.
• PP-DPIN shows superior performance and robustness against attacks compared to existing methods.