Toward the Flatter Landscape and Better Generalization in Federated Learning Under Client-Level Differential Privacy.

Area of Science:

• Machine Learning
• Cybersecurity
• Distributed Systems

Background:

• Federated Learning (FL) enables collaborative model training without sharing raw data.
• Client-level Differentially Private FL (DPFL) is standard for privacy but causes performance degradation due to noisy updates.
• Existing DPFL methods struggle with sharp loss landscapes and poor weight perturbation robustness.

Purpose of the Study:

• To propose novel DPFL algorithms that mitigate performance degradation caused by differential privacy.
• To enhance the stability and robustness of DPFL models against noise and perturbations.
• To achieve state-of-the-art performance in private federated learning.

Main Methods:

• Introduced DP-FedSAM, integrating the Sharpness Aware Minimization (SAM) optimizer into DPFL.
• Developed DP-FedSAM-topk, incorporating local update sparsification for reduced noise magnitude.
• Conducted theoretical analysis including convergence, Rényi DP, sensitivity, and generalization.

Main Results:

• DP-FedSAM generates flatter, more stable local models, improving robustness to DP noise and perturbations.
• DP-FedSAM-topk further enhances performance by sparsifying local updates, reducing noise impact.
• Theoretical analyses confirm the algorithms' ability to mitigate DP-induced performance degradation.
• Empirical results demonstrate state-of-the-art performance compared to existing DPFL baselines.

Conclusions:

• DP-FedSAM and DP-FedSAM-topk effectively address the performance degradation issues in DPFL.
• The proposed methods offer improved stability, robustness, and privacy guarantees.
• These algorithms represent a significant advancement in achieving high-performance, private federated learning.