FedGAMMA: Federated Learning With Global Sharpness-Aware Minimization.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated learning (FL) enables privacy-preserving distributed training but suffers from client drift caused by data heterogeneity and partial client participation.
• Existing methods primarily focus on empirical risk minimization, overlooking the impact of global loss landscape geometry on generalization.
• Client drift leads to divergence between local and global updates, degrading model performance.

Purpose of the Study:

• To propose FedGAMMA, a novel federated learning algorithm designed to address client drift and improve generalization.
• To investigate the relationship between global loss landscape flatness and model generalization in federated learning.
• To develop a method that aligns local client updates towards a shared global objective, promoting a flatter global landscape.

Main Methods:

• FedGAMMA employs Global sharpness-Aware MiniMizAtion (GAMMA) to optimize for a flat global loss landscape.
• A local varieties control technique is introduced to align client updates and mitigate client drift.
• The algorithm contrasts with FedSAM by focusing on global flatness rather than solely local flatness.

Main Results:

• FedGAMMA significantly outperforms existing federated learning baselines across various datasets.
• The proposed algorithm effectively addresses the client drift issue inherent in federated learning.
• FedGAMMA successfully promotes a smoother and flatter global loss landscape, enhancing generalization.

Conclusions:

• FedGAMMA offers a robust solution to the client drift problem in federated learning.
• Optimizing for a flat global landscape is crucial for achieving high performance and generalization in FL.
• The local varieties control technique in FedGAMMA effectively aligns clients towards a unified, flatter global objective.