Automatically Design Convolutional Neural Networks by Optimization With Submodularity and Supermodularity.

Area of Science:

• Artificial Intelligence
• Computer Science
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) are powerful deep learning models, but their performance is highly dependent on architecture.
• Intelligently designing CNN architectures to meet specific performance goals remains a significant challenge in machine learning.

Purpose of the Study:

• To address the practical problems of maximizing accuracy within a given time constraint and minimizing time for a target accuracy.
• To develop efficient algorithms for automated CNN architecture optimization.

Main Methods:

• Formulating architecture optimization as submodular optimization problems by incorporating prior knowledge.
• Proposing and analyzing efficient Greedy algorithms to solve these submodular problems, including theoretical bounds.

Main Results:

• The proposed Greedy algorithms demonstrate efficiency in solving CNN architecture optimization problems.
• Experimental results on public datasets show competitive or superior performance compared to other hyperparameter optimization methods.

Conclusions:

• The novel approach effectively converts CNN architecture design into solvable submodular optimization problems.
• The developed Greedy algorithms offer an efficient and effective solution for optimizing CNN architectures for accuracy and speed.