FedNK-RF: Federated Kernel Learning With Heterogeneous Data and Optimal Rates.

Area of Science:

• Machine Learning
• Decentralized Systems
• Data Science

Background:

• Federated learning (FL) is a privacy-preserving decentralized learning approach.
• Data heterogeneity in FL can negatively impact predictive accuracy and generalization analysis.
• Kernel learning methods offer powerful tools for analyzing complex data relationships.

Purpose of the Study:

• To propose efficient federated kernel learning (FedK) algorithms.
• To analyze the generalization properties of these FedK algorithms.
• To address the challenges posed by data heterogeneity in FL.

Main Methods:

• Federated Kernel Learning with Random Features (FedK-RF): Shares random features of local data subsets for global information acquisition.
• Federated Nyström Approximation with Random Features (FedNK-RF): Builds upon FedK-RF to reduce approximation errors.
• Integral Operator Theory: Used to derive excess risk bounds and analyze generalization.

Main Results:

• FedK-RF enhances predictive capability while preserving privacy.
• FedNK-RF further reduces errors compared to FedK-RF.
• Derived excess risk bounds quantify the impact of data heterogeneity and shared information.
• Experimental results validate the superiority of FedNK-RF.

Conclusions:

• The proposed FedK algorithms effectively handle data heterogeneity in federated learning.
• FedNK-RF offers an improved approach for federated kernel learning with strong generalization guarantees.
• The theoretical analysis provides insights into the trade-offs between data heterogeneity and information sharing.