Efficient and accurate compound scaling for convolutional neural networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Convolutional Neural Networks (ConvNets) are increasingly vital for diverse workloads, necessitating efficient and accurate network architectures.
• Current ConvNet scaling methods often ignore inter-dimensional relationships and inference speed impacts, leading to suboptimal accuracy-inference speed trade-offs.

Purpose of the Study:

• To propose a novel scaling method for ConvNets that improves both accuracy and inference speed.
• To address the limitations of existing scaling techniques by incorporating dimension relationships and runtime proxy constraints.

Main Methods:

• Empirically quantifying the relationship between convolutional width and input resolution, noting filter redundancy at higher resolutions.
• Developing a runtime prediction model that considers fine-grained layer computational properties for efficient network configuration.
• Systematically adjusting network dimensions (width, depth, resolution) based on quantified relationships and runtime predictions.

Main Results:

• The proposed scaling method demonstrates superior performance compared to prior works on ImageNet datasets.
• Achieved a better trade-off between accuracy and inference speed across various ConvNet architectures.
• Generated a set of models with improved parametric efficiency.

Conclusions:

• The novel scaling strategy effectively balances accuracy and inference speed by leveraging dimension relationships and runtime predictions.
• This method provides a more efficient way to adapt ConvNets for diverse computational demands.
• The findings offer a significant advancement in designing high-performance neural network architectures.