Allosteric Feature Collaboration for Model-Heterogeneous Federated Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated learning (FL) typically assumes homogeneous client models, limiting its application.
• Model heterogeneity in FL (M-hete FL) presents challenges due to the inability to directly aggregate diverse client model parameters.

Purpose of the Study:

• To propose a novel method, Allosteric Feature Collaboration (AlFeCo), for effective collaborative learning in model-heterogeneous federated learning settings.
• To address the challenges of parameter aggregation in M-hete FL by enabling knowledge interchange across structurally different client models.

Main Methods:

• Developed an allosteric feature generator to extract task-relevant information from multiple client models.
• Utilized client-shared and client-specific codes for knowledge interchange and generation of dimensionally variable allosteric features.
• Implemented a dual-path communication mechanism (model-model and model-prediction) to supervise collaborative model updates using allosteric features.

Main Results:

• AlFeCo effectively facilitates knowledge interchange between heterogeneous client models.
• The proposed method demonstrates strong performance on classical FL benchmarks.
• AlFeCo proves effective in model-heterogeneous federated anti-spoofing tasks.

Conclusions:

• AlFeCo offers a viable solution for collaborative learning in model-heterogeneous federated learning environments.
• The method enhances communication and knowledge sharing between diverse client models.
• Theoretical evidence and convergence analysis support the effectiveness of AlFeCo in M-hete FL.