Non-IID Federated Learning With Sharper Risk Bound

Area of Science:

• Machine Learning
• Distributed Systems

Background:

• Federated learning (FL) faces performance degradation due to non-independently and identically distributed (non-IID) data partitioning.
• Existing research on non-IID FL has focused on algorithms and optimization, with limited theoretical analysis of generalization performance.

Purpose of the Study:

• To develop effective analytical tools for understanding the generalization properties of non-IID FL.
• To propose a novel framework that improves the generalization performance of FL models under non-IID data.

Main Methods:

• Introduction of weighted local Rademacher complexity as a novel analytical approach for non-IID FL generalization.
• Derivation of a sharper excess risk bound utilizing the proposed weighted local Rademacher complexity.
• Development of a general framework, federated averaging with local rademacher complexity (FedALRC), based on theoretical findings.

Main Results:

• A faster convergence rate for excess risk bounds compared to existing methods.
• FedALRC demonstrates superior performance over established methods like FedAvg, FedProx, and FedNova in experimental evaluations.
• Experimental results align with and validate the theoretical findings.

Conclusions:

• Weighted local Rademacher complexity provides an effective tool for analyzing non-IID FL generalization.
• The FedALRC framework successfully reduces excess risk without increasing communication costs.
• The proposed methods offer a promising direction for improving FL performance in heterogeneous data environments.