Resilient and Communication Efficient Learning for Heterogeneous Federated Systems

Area of Science:

• Artificial Intelligence
• Machine Learning
• Edge Computing

Background:

• Federated Learning (FL) enables machine learning on decentralized data from edge devices.
• Heterogeneous network topologies and wireless transmission uncertainty hinder FL's efficiency and increase convergence time and communication costs in edge computing.

Purpose of the Study:

• To propose a novel Federated Learning (FL) scheme that addresses system heterogeneity and wireless connection uncertainty in edge computing.
• To enhance the convergence speed and communication efficiency of FL in challenging edge environments.

Main Methods:

• Edge devices learn self-distilled neural networks, which are prunable to various sizes.
• The proposed method allows partial transmission of model parameters even with faulty network connections, preserving knowledge.

Main Results:

• The approach effectively handles system heterogeneity by supporting diverse model architectures on edge devices.
• It demonstrates significant resilience and improved communication efficiency under network instability compared to existing methods.

Conclusions:

• The developed FL scheme offers a robust solution for deploying machine learning in heterogeneous and unreliable edge computing environments.
• Self-distilled, prunable neural networks are key to overcoming the limitations of traditional FL in edge applications.