A Communication-Efficient, Privacy-Preserving Federated Learning Algorithm Based on Two-Stage Gradient Pruning and...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Cybersecurity

Background:

• Federated learning (FL) faces challenges in security and high communication costs.
• Differential privacy (DP) protects data but can reduce model accuracy.
• Existing model pruning methods aim to reduce communication overhead.

Purpose of the Study:

• To develop a communication-efficient and privacy-preserving FL algorithm.
• To address the trade-off between privacy protection and model accuracy in FL.
• To reduce the communication burden in training large-scale federated models.

Main Methods:

• Introduced IsmDP-FL, a two-stage federated learning algorithm.
• Employed gradient pruning and differentiated differential privacy.
• Applied DP to important parameters after pruning and to remaining parameters in network layers.

Main Results:

• Demonstrated high communication efficiency in extensive experiments.
• Successfully maintained model privacy throughout the federated learning process.
• Reduced the unnecessary consumption of the privacy budget.

Conclusions:

• IsmDP-FL effectively balances privacy and communication efficiency in federated learning.
• The proposed method offers a practical solution for secure and efficient large-scale model training.
• This approach minimizes privacy budget usage without compromising model performance.