Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Federated learning (FL) enables collaborative model training under privacy constraints.
• Divergent data distributions across clients often lead to suboptimal FL performance.
• Existing federated multitask learning (FMTL) approaches may require protocol modifications or prior knowledge of cluster numbers.

Purpose of the Study:

• To introduce Clustered Federated Learning (CFL), a novel FMTL framework.
• To address performance degradation in FL caused by heterogeneous client data distributions.
• To provide a flexible, privacy-preserving, and mathematically guaranteed clustering solution for FL.

Main Methods:

• CFL exploits geometric properties of the FL loss surface for client grouping.
• It groups clients into clusters with jointly trainable data distributions.
• Clustering is performed as a postprocessing step after FL convergence.

Main Results:

• CFL achieves greater or equal performance compared to conventional FL.
• It allows for the development of more specialized models for client groups.
• Experiments with deep convolutional and recurrent neural networks validate the theoretical analysis.

Conclusions:

• CFL offers a significant improvement over standard FL for non-identically distributed data.
• The framework is applicable to general non-convex objectives, including deep neural networks.
• CFL enhances FL by enabling personalized models through effective client clustering.