Divide-and-conquer the NAS puzzle in resource-constrained federated learning systems

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated Learning (FL) enables privacy-preserving machine learning on edge devices.
• Designing efficient neural architectures in FL systems remains a challenge.
• Current methods often overlook overall system efficiency in federated neural architecture search.

Purpose of the Study:

• To propose DC-NAS, a novel divide-and-conquer approach for efficient Neural Architecture Search (NAS) in federated environments.
• To introduce a diversified sampling strategy balancing exploration and exploitation for systematic search space sampling.
• To reduce computational complexity at edge devices through channel pruning.

Main Methods:

• Implementing a supernet-based NAS within a federated system.
• Utilizing a novel diversified sampling strategy that dynamically adjusts sample distances.
• Applying channel pruning to decrease training complexity on devices.
• Evaluating performance on CIFAR10, CIFAR100, EMNIST, and TinyImagenet benchmarks.

Main Results:

• DC-NAS outperforms existing sampling strategies, including Hadamard sampling.
• Demonstrates comprehensive analysis of scalability and non-IID data handling in FL.
• Achieves near iso-accuracy compared to full-scale federated NAS.
• Reduces resource requirements by 50%.

Conclusions:

• DC-NAS offers an efficient solution for neural architecture design in federated learning.
• The proposed sampling strategy and channel pruning significantly improve system efficiency.
• This approach provides a practical method for deploying NAS in resource-constrained federated environments.