FedTKD: A Trustworthy Heterogeneous Federated Learning Based on Adaptive Knowledge Distillation.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Privacy-Preserving Technologies

Background:

• Traditional federated learning (FL) requires homogeneous model structures, limiting its application in real-world heterogeneous environments.
• Existing knowledge distillation methods for heterogeneous FL often assume client trustworthiness, failing to address malicious or low-quality data contributions.
• Integrating personalized models in FL is challenging due to model heterogeneity and the need for trustworthy knowledge aggregation.

Purpose of the Study:

• To propose a trustworthy heterogeneous federated learning framework (FedTKD) addressing client identification and reliable knowledge fusion.
• To enable federated learning in environments with diverse client model structures and potential malicious participants.
• To enhance the accuracy and robustness of federated models under heterogeneous conditions.

Main Methods:

• Developed a malicious client identification method using client logit features to filter unreliable information.
• Implemented a selective knowledge fusion technique for high-quality global logit computation.
• Introduced an adaptive knowledge distillation method for improved server-to-client knowledge transfer.

Main Results:

• FedTKD demonstrated superior performance compared to baseline methods across various attack and data distribution scenarios.
• The framework exhibited stable performance even under different attack strategies.
• Achieved a 2% to 3% accuracy improvement in federated models with heterogeneous data distributions.

Conclusions:

• FedTKD effectively addresses the challenges of trustworthy knowledge fusion in heterogeneous federated learning environments.
• The proposed methods for client identification and selective knowledge aggregation enhance model reliability and privacy.
• This framework offers a robust solution for practical federated learning applications with diverse and potentially untrustworthy clients.