Dense Optimizer: An Information Entropy-Guided Structural Search Method for Dense-Like Neural Network Design.

Area of Science:

• Computer Vision
• Deep Learning Architectures
• Machine Learning Optimization

Background:

• Dense convolutional networks (DenseNets) offer efficient and compact architectures.
• Manual design of DenseNets faces challenges in channel adjustment and reuse.
• Automating DenseNet architecture search is crucial for further performance gains.

Purpose of the Study:

• To propose an automated architecture search method for high-performance dense-like networks.
• To develop a novel optimization framework for efficient network design.
• To enhance accuracy and reduce computational cost in computer vision models.

Main Methods:

• Developed 'dense optimizer,' an architecture search method for dense networks.
• Treated dense networks as hierarchical information systems, maximizing information entropy.
• Utilized a power law to constrain entropy distribution and formulated an optimization problem.
• Implemented a branch-and-bound algorithm integrating power-law principles and search space scaling.

Main Results:

• Validated 'dense optimizer' on diverse computer vision benchmark datasets.
• Achieved 84.3% top-1 accuracy on CIFAR-100 with the searched DenseNet-OPT model, a 5.97% improvement.
• Demonstrated high-quality search results with only 4 hours of computation time on a single CPU.

Conclusions:

• 'Dense optimizer' successfully automates the design of high-performance dense convolutional networks.
• The method significantly improves accuracy compared to manually designed counterparts.
• Efficient search process makes it practical for real-world applications.