FedMPS: Federated Learning in a Synergy of Multi-Level Prototype-Based Contrastive Learning and Soft Label Generation

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Federated learning (FL) enables collaborative model training without sharing raw data, preserving user privacy.
• Data heterogeneity across clients in FL introduces bias, degrading local model performance and slowing convergence.
• Current FL methods face challenges in efficient knowledge transfer, leading to high communication overhead and suboptimal results.

Purpose of the Study:

• To propose a novel Federated Learning framework, FedMPS, that addresses data heterogeneity and communication efficiency.
• To enhance collaborative learning by integrating multi-level prototype-based contrastive learning and soft label generation.
• To reduce global knowledge shift and communication costs in FL.

Main Methods:

• Constructing multi-level prototypes from different model layers to capture both high-level semantics and low-level details.
• Utilizing contrastive learning (CL) with these prototypes to improve feature space discriminability and consistency.
• Introducing a prototype-guided soft label generation module to model inter-class relationships in the output space.

Main Results:

• FedMPS effectively reduces communication costs by transmitting only prototypes and soft labels, not model parameters.
• The proposed method demonstrates improved intra-class discriminability and consistency in the feature space.
• Experimental results on six datasets show FedMPS outperforms state-of-the-art FL approaches.

Conclusions:

• FedMPS offers an effective solution for federated learning under data heterogeneity.
• The framework achieves better performance and efficiency compared to existing FL methods.
• Transmitting prototypes and soft labels is a viable strategy for knowledge sharing in FL.