FedHyperGraph: A layer-wise personalized federated learning with correlation graphs in hyperbolic space.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Distributed Systems

Background:

• Personalized Federated Learning (PFL) aims for model personalization in heterogeneous data scenarios.
• Current PFL methods often aggregate model parameters monolithically, ignoring crucial cross-client parameter correlations.
• This coarse-grained approach limits personalization effectiveness when clients have similar tasks but diverse data.

Purpose of the Study:

• To propose FedHyperGraph, a graph-guided aggregation framework for layer-wise PFL.
• To address the limitations of monolithic parameter aggregation in existing PFL approaches.
• To leverage latent correlations among parameters across clients for enhanced personalization.

Main Methods:

• Developed FedHyperGraph, a framework utilizing layer-wise knowledge.
• Constructed latent correlation graphs in hyperbolic space to guide aggregation.
• Implemented a graph-guided aggregation process for personalized model updates.

Main Results:

• FedHyperGraph achieved significant accuracy improvements: up to 42.6% on graph, 33.1% on computer vision (CV), and 7.5% on natural language processing (NLP) datasets.
• Demonstrated accelerated convergence by up to 52.6% on CV tasks.
• Showcased superior scalability across various client scales.

Conclusions:

• FedHyperGraph effectively captures layer-wise knowledge and latent correlations for superior PFL.
• The hyperbolic space representation enhances the personalized aggregation process.
• FedHyperGraph offers substantial improvements in accuracy, convergence, and scalability for PFL.