Towards Straggler-Resilient Split Federated Learning: An Unbalanced Update Approach.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Split Federated Learning (SFL) combines Federated Learning (FL) and Split Learning (SL) for scalable edge device training.
• SFL is hindered by the 'straggler issue,' where slow clients create significant latency.
• Client-server dependency in SFL exacerbates delays, bottlenecking system efficiency.

Purpose of the Study:

• To propose a straggler-resilient Split Federated Learning algorithm.
• To decouple training progress from straggler delays in SFL systems.
• To enhance the scalability and efficiency of SFL by mitigating straggler impact.

Main Methods:

• Introduced MU-SplitFed, a zeroth-order optimization algorithm for SFL.
• Implemented an unbalanced update mechanism allowing the server to perform τ local updates per client round.
• Analyzed convergence rate for non-convex objectives as 𝒪(√(d/(τT))) with linear speedup in τ communication rounds.

Main Results:

• MU-SplitFed demonstrates resilience to straggler delays in SFL.
• Achieved a convergence rate showing linear speedup with respect to the number of server updates (τ).
• Experimental results confirm superior performance over baseline methods in the presence of stragglers.

Conclusions:

• MU-SplitFed effectively mitigates the straggler bottleneck in SFL.
• The adaptive tuning of τ allows for effective management of straggler impact.
• The proposed method enhances the practical applicability of SFL in real-world distributed systems.