Nonconvex Federated Composite Optimization With Random Reshuffling and Biased Compression.

Area of Science:

• Machine Learning
• Optimization Theory
• Distributed Computing

Background:

• Federated learning (FL) enables collaborative model training on decentralized data.
• Nonconvex composite optimization problems present significant challenges in FL due to complex loss landscapes and nonsmooth regularizers.
• Existing FL algorithms often struggle with efficiency and convergence in these challenging settings.

Purpose of the Study:

• To propose FedRREF, a novel federated learning algorithm designed to address nonconvex federated composite optimization (FCO) problems.
• To reduce computation and communication costs in federated learning through algorithmic innovation.
• To establish a theoretical convergence rate for the proposed algorithm in nonsmooth and nonconvex settings.

Main Methods:

• Integration of error feedback (EF) with the random reshuffling (RR) technique.
• Development of FedRREF algorithm tailored for nonsmooth and nonconvex federated composite optimization.
• Theoretical analysis to determine the convergence rate of FedRREF.

Main Results:

• FedRREF achieves a convergence rate of $\mathcal{O}(1/\sqrt{T})$ communication rounds.
• The algorithm demonstrates reduced computation and communication overhead compared to existing methods.
• Numerical experiments validate the effectiveness and practical applicability of FedRREF.

Conclusions:

• FedRREF offers an efficient and effective solution for nonconvex federated composite optimization.
• The simultaneous consideration of random reshuffling and biased compression is novel in nonsmooth and nonconvex FL.
• The proposed algorithm shows promise for real-world applications requiring efficient distributed learning on complex data.