NGDE: A Niching-Based Gradient-Directed Evolution Algorithm for Nonconvex Optimization.

Area of Science:

• Machine Learning
• Optimization Algorithms
• Data Science

Background:

• Nonconvex optimization is common in machine learning and data science.
• Gradient-based methods risk local optima; evolutionary algorithms are computationally intensive.
• High-dimensional problems require efficient global optimization strategies.

Purpose of the Study:

• Introduce a novel algorithm, niching-based gradient-directed evolution (NGDE), for high-dimensional nonconvex optimization.
• Address limitations of existing gradient-based and evolutionary algorithms.
• Improve performance in complex optimization tasks.

Main Methods:

• NGDE generates solutions and divides them into niches for diverse exploration.
• A gradient-directed mutation operator creates candidate offspring.
• Convergence is analyzed using full gradients and mini-batch approximations.

Main Results:

• NGDE demonstrates superior performance in minimizing multimodal functions.
• The algorithm achieves significantly improved classification accuracy in LeNet-5 neural networks, especially with smaller training datasets.
• Effective exploration of distinct feasible region areas.

Conclusions:

• NGDE offers an effective approach for high-dimensional nonconvex optimization.
• The algorithm balances exploration and exploitation for improved global search.
• NGDE shows promise for enhancing machine learning model training and performance.