FedEff: efficient federated learning with optimal local epochs for heterogeneous clients

Area of Science:

• Machine Learning
• Distributed Systems
• Artificial Intelligence

Background:

• Federated Learning (FL) enables collaborative training without data centralization.
• System and statistical heterogeneity across clients often degrade FL efficiency.
• Increasing local epochs can improve efficiency but risks model divergence and slower convergence.

Purpose of the Study:

• To analyze the trade-off between local epochs and model divergence in FL.
• To propose an efficient FL algorithm (FedEff) addressing heterogeneity.
• To reduce client waiting times and overall training duration.

Main Methods:

• Empirical divergence analysis to understand the local epoch trade-off.
• Development of FedEff, a server-side epoch selection mechanism.
• Utilizing Estimated Round Time (ERT) to determine optimal local epochs per client based on client speeds.

Main Results:

• Consistent local updates reduce mean divergence, promoting stable convergence.
• FedEff achieves notable reductions in client waiting times and training duration.
• FedEff outperforms FedAvg and random epoch selection in heterogeneous settings.

Conclusions:

• Optimizing local epochs is crucial for efficient FL under heterogeneity.
• FedEff effectively balances local training and global model convergence.
• The proposed algorithm enhances federated learning performance and efficiency.