Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated learning (FL) enables collaborative model training without raw data sharing.
• Participant resource heterogeneity (hardware, inference speed) challenges existing FL methods.
• Need for adaptable models that meet diverse inference requirements.

Purpose of the Study:

• To propose a novel Split-Mix FL strategy for heterogeneous participants.
• To enable on-demand customization of model size and robustness post-training.
• To improve the efficiency and applicability of FL in dynamic environments.

Main Methods:

• Learning a set of base sub-networks with varying sizes and robustness levels.
• On-demand aggregation of sub-networks tailored to specific inference needs.
• Implementing the Split-Mix FL strategy for heterogeneous resource environments.

Main Results:

• The proposed Split-Mix FL strategy achieves efficient in-situ customization.
• Demonstrated superior performance compared to existing heterogeneous-architecture FL methods.
• Validated high efficiency in communication, storage, and inference.

Conclusions:

• Split-Mix FL effectively addresses heterogeneity and dynamics in practical FL scenarios.
• Enables flexible and efficient model adaptation for diverse inference requirements.
• Significantly broadens the applicability of federated learning.