Communication-Efficient Split Learning via Adaptive Feature-Wise Compression

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Computing

Background:

• Split learning (SL) enables collaborative model training across decentralized devices.
• High communication overhead between clients and servers limits SL efficiency.
• Existing SL methods struggle to balance communication reduction and model accuracy.

Purpose of the Study:

• To propose SplitFC, a novel communication-efficient split learning framework.
• To reduce the communication burden of transmitting intermediate features and gradients in SL.
• To maintain high model accuracy while significantly decreasing communication costs.

Main Methods:

• Leveraging matrix column dispersion for compression.
• Implementing adaptive feature-wise dropout based on vector standard deviation.
• Applying adaptive feature-wise quantization with closed-form optimal levels.
• Utilizing the chain rule to drop corresponding gradient vectors.

Main Results:

• SplitFC demonstrated substantial reductions in communication overhead.
• The framework maintained high prediction accuracy across multiple datasets (MNIST, CIFAR-100, CelebA).
• SplitFC outperformed existing state-of-the-art split learning frameworks.

Conclusions:

• SplitFC offers an effective solution for communication-efficient split learning.
• The proposed adaptive compression strategies significantly enhance SL performance.
• This framework paves the way for more practical and scalable distributed machine learning.