Federated Learning With Taskonomy for Non-IID Data.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Classical federated learning struggles with non-independent and identically distributed (non-IID) client data, leading to performance degradation.
• Existing clustering methods for non-IID data are often slow and face convergence challenges.

Purpose of the Study:

• To introduce Federated Learning with Taskonomy (FLT), a novel method for efficient federated aggregation of heterogeneous data.
• To generalize existing clustering approaches by learning task relatedness between clients.

Main Methods:

• Clients compress data into latent representations using a pretrained encoder and send data signatures to the server.
• The server employs manifold learning to understand client task relatedness and applies a generalized federated averaging.
• FLT supports generic client relatedness graphs and can decompose them into clusters.

Main Results:

• FLT significantly outperforms state-of-the-art baselines in non-IID federated learning settings.
• The proposed method demonstrates improved fairness among clients.
• FLT offers a one-off process for learning task relatedness, avoiding iterative convergence issues.

Conclusions:

• Federated Learning with Taskonomy (FLT) provides an efficient and effective solution for handling heterogeneous data in federated learning.
• FLT enhances model performance and fairness, offering a flexible approach to client relatedness.
• The codebase is publicly available for further research and application.