Stabilizing and Accelerating Federated Learning on Heterogeneous Data With Partial Client Participation.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated learning (FL) enables collaborative model training without centralizing data.
• Multiple local updates per client reduce communication overhead but can destabilize global convergence.
• Data heterogeneity across clients is a primary driver of instability in FL.

Purpose of the Study:

• To introduce a novel local-update stability index for federated learning.
• To analyze the impact of local updates on FL stability and convergence.
• To propose an accelerated and stabilized FL algorithm.

Main Methods:

• Defined a local-update stability index based on inter-client model discrepancy.
• Theoretically analyzed stability for state-of-the-art FL methods.
• Developed FedANAG, an FL algorithm using server- and client-level Nesterov accelerated gradient (NAG).

Main Results:

• The local-update stability index quantifies the influence of client model variation on global models.
• FedANAG enhances local update stability using global momentum.
• FedANAG demonstrates accelerated convergence and higher accuracy across various data heterogeneity and participation ratios.

Conclusions:

• Local updates in FL can negatively impact stability and convergence, especially with data heterogeneity.
• The proposed stability index provides insights into FL method limitations.
• FedANAG offers a promising solution for stable and efficient federated learning.