A Gradient-Guided Evolutionary Approach to Training Deep Neural Networks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Optimization

Background:

• Efficient training of neural networks (NNs) is vital for classification performance.
• Gradient-based methods for NN training can get stuck in local optima and are sensitive to hyperparameters.
• Evolutionary algorithms (EAs) offer robustness but struggle with high-dimensional problems and deep neural networks (DNNs).

Purpose of the Study:

• To propose a novel hybrid approach for training deep neural networks (DNNs).
• To leverage the strengths of both gradient-based methods and evolutionary algorithms (EAs).
• To enhance the efficiency and performance of DNN training, particularly for large-scale optimization problems.

Main Methods:

• A gradient-guided evolutionary approach is proposed to train DNNs.
• A novel genetic operator optimizes weights using gradient information for search direction.
• Network sparsity is incorporated to reduce complexity and mitigate overfitting.

Main Results:

• The proposed approach demonstrates effectiveness across various neural network architectures, including single-layer, deep-layer, recurrent, and convolutional neural networks (CNNs).
• The method successfully trains DNNs by integrating gradient information into the evolutionary search process.
• Network sparsity effectively reduces model complexity and alleviates overfitting issues.

Conclusions:

• The developed gradient-guided evolutionary approach offers a powerful new method for training DNNs.
• This hybrid strategy enhances the applicability of EAs to complex, large-scale optimization problems.
• The approach improves DNN training efficiency and classification performance while managing network complexity.