Feature Correlation-Guided Knowledge Transfer for Federated Self-Supervised Learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Self-supervised learning (SSL) is applied to federated learning (FL) to address data labeling challenges.
• Existing federated SSL (FedSSL) methods often assume homogeneous models or require public datasets, limiting their general applicability.
• Heterogeneous models and unlabeled clients present significant hurdles for universal FedSSL frameworks.

Purpose of the Study:

• To propose a novel and general method, Federated Self-Supervised Learning with Feature-Correlation-based Aggregation (FedFoA), for training in heterogeneous federated environments.
• To overcome the limitations of existing FedSSL approaches that rely on parameter or feature sharing.
• To enable effective knowledge transfer and collaboration among unlabeled clients with heterogeneous models.

Main Methods:

• FedFoA exchanges feature correlations instead of model parameters or feature mappings to reduce discrepancies in local representation learning.
• A factorization-based method extracts a cross-feature relation matrix from local representations, serving as a knowledge medium for aggregation.
• The framework is designed to be heterogeneity-supportive, privacy-preserving, and compatible with existing FedSSL methods.

Main Results:

• FedFoA effectively reduces discrepancies in local representation learning processes.
• The proposed method promotes collaboration between heterogeneous clients.
• Extensive experiments show FedFoA significantly outperforms state-of-the-art methods.

Conclusions:

• FedFoA offers a general and effective solution for federated self-supervised learning, particularly in heterogeneous settings.
• The feature-correlation-based aggregation approach enhances collaboration and performance without strong assumptions on client models.
• The method demonstrates significant improvements over existing approaches, highlighting its potential for real-world applications.