Improving Generalization and Personalization in Model-Heterogeneous Federated Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Conventional federated learning (FL) assumes homogeneous client models, requiring parameter sharing which poses security risks and neglects personalization.
• Existing FL methods struggle to balance server model generalization with individual client personalization, especially with heterogeneous models.

Purpose of the Study:

• To address the challenge of ensuring both generalization and personalization in federated learning with heterogeneous client models.
• To introduce a novel federated learning framework, FedTED, capable of handling diverse client models and objectives.

Main Methods:

• FedTED utilizes a twin-branch structure to manage heterogeneous models.
• Data-free knowledge distillation (DFKD) is employed to facilitate knowledge transfer without raw data.
• The framework coordinates updates from heterogeneous clients to build a robust global model.

Main Results:

• FedTED significantly improves both personalization and generalization performance.
• Achieved a 19.37% enhancement in generalization and up to 9.76% improvement in personalization.
• Demonstrated superior performance over representative algorithms in heterogeneous FL scenarios.

Conclusions:

• FedTED effectively addresses the limitations of conventional FL in heterogeneous environments.
• The proposed framework successfully balances the competing goals of global generalization and local client personalization.
• FedTED offers a promising solution for more effective and secure federated learning applications.