Top-k Shuffled Differential Privacy Federated Learning for Heterogeneous Data

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Privacy

Background:

• Federated learning (FL) enables collaborative model training without centralizing sensitive data.
• Key challenges in FL include data heterogeneity, privacy vulnerabilities, and high communication overhead.
• Existing privacy-preserving FL methods struggle with heterogeneous data, leading to increased communication costs.

Purpose of the Study:

• To propose a novel federated learning framework, TopkSDP-FL, designed for heterogeneous data environments.
• To enhance privacy guarantees and reduce communication overhead in FL.
• To mitigate model drift in decentralized training scenarios.

Main Methods:

• Development of a TopkSDP-FL framework incorporating a bidirectional top-k communication mechanism.
• Introduction of a contrastive learning-inspired regularization for local training to combat model drift.
• Implementation of layer-level gradient parameter shuffling for enhanced privacy and budget management.
• Formal privacy analysis to confirm privacy amplification effects.

Main Results:

• TopkSDP-FL demonstrated superior performance compared to state-of-the-art FL methods, particularly in non-IID (non-independently and identically distributed) scenarios.
• Achieved an average accuracy improvement of 3% over FedAvg and other leading algorithms.
• Reduced communication costs by over 90% while enhancing privacy protection.

Conclusions:

• The proposed TopkSDP-FL framework effectively addresses data heterogeneity and privacy concerns in federated learning.
• The novel methods significantly improve efficiency by reducing communication overhead.
• TopkSDP-FL offers a robust solution for secure and efficient decentralized AI model training.