A Progressive Subnetwork Searching Framework for Dynamic Inference.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural network (DNN) model compression is crucial for efficient hardware acceleration but often results in fixed models.
• Existing compressed models lack the flexibility to adapt computing complexity (latency) to dynamic requirements.

Purpose of the Study:

• To develop a dynamic DNN framework capable of run-time adaptation of computing structures.
• To improve the accuracy and flexibility of compressed DNNs for diverse hardware and workloads.

Main Methods:

• Proposed a progressive subnetwork searching framework for constructing dynamic DNNs.
• Introduced novel techniques: trainable noise ranking, channel-group sampling, selective fine-tuning, and subnet filtering.
• Trained dynamic DNNs using a cross-entropy objective function with multiple subnets sampled from a supernet.

Main Results:

• Achieved higher accuracy for all subnets compared to prior works on CIFAR-10 and ImageNet datasets.
• Demonstrated accuracy gains on ImageNet: 0.9% (Alexnet), 2.5% (ResNet18), 1.1% (VGG11), and 0.58% (MobileNetv1) compared to US-NN.
• Successfully deployed dynamic networks on Nvidia GPUs and Intel CPUs, showing significant improvements in run-time latency tuning.

Conclusions:

• The proposed progressive subnetwork searching framework effectively constructs high-quality dynamic DNNs.
• Dynamic DNNs offer adaptable computing complexity, enhancing performance and accuracy on real-world hardware.
• This work advances the field of efficient and adaptable deep learning models.