Structure Learning for Deep Neural Networks Based on Multiobjective Optimization.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep neural networks (DNNs) often contain numerous connecting parameters, potentially hindering generalization.
• Optimizing the internal structure of DNNs is crucial for improving model performance and interpretability.

Purpose of the Study:

• To develop a layerwise structure learning method for DNNs that balances representation ability and network sparsity.
• To enhance the generalization capabilities of deep learning models by reducing redundant connecting parameters.

Main Methods:

• A layerwise structure learning approach based on multiobjective optimization is proposed.
• The method models visible data using products of experts (PoE) and incorporates connecting sparsity as an objective.
• An improved multiobjective evolutionary algorithm is employed to solve the established model, with computational cost reduction techniques.

Main Results:

• Experiments at single-layer, hierarchical, and application levels validate the proposed algorithm's effectiveness.
• The learned network structures demonstrate improved performance in deep neural networks.
• The method successfully balances representation ability and network connecting sparsity.

Conclusions:

• The proposed layerwise structure learning method effectively optimizes DNNs for better generalization.
• Reducing connecting parameters through multiobjective optimization is a viable strategy for enhancing DNN performance.
• The approach offers a novel way to learn optimal structures for deep learning models.