Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Federated learning enables collaborative model training on decentralized private data.
• Model heterogeneity and catastrophic forgetting are key limitations in federated learning.

Purpose of the Study:

• To introduce FCCL+, a novel federated learning approach to tackle heterogeneity and catastrophic forgetting.
• To enhance both intra-domain discriminability and inter-domain generalization in federated learning.

Main Methods:

• Leveraging irrelevant unlabeled public data to bridge communication gaps between heterogeneous participants.
• Constructing cross-correlation matrices and aligning instance similarity distributions at logit and feature levels.
• Implementing Federated Non-Target Distillation to retain inter-domain knowledge and prevent optimization conflicts.

Main Results:

• FCCL+ effectively overcomes communication barriers caused by model heterogeneity.
• The method improves generalizable ability by retaining inter-domain knowledge.
• Empirical results demonstrate the superiority and module efficiency of FCCL+ across various scenarios.

Conclusions:

• FCCL+ offers a robust solution for heterogeneous federated learning challenges.
• The proposed benchmark facilitates standardized evaluation of federated learning methods.
• The approach significantly advances the applicability and generalizability of federated learning.