FBCA: Flexible Besiege and Conquer Algorithm for Multi-Layer Perceptron Optimization Problems

Area of Science:

• Artificial Intelligence
• Machine Learning
• Optimization Algorithms

Background:

• Multi-Layer Perceptrons (MLPs) are fundamental to deep learning models like CNNs, RNNs, and Transformers.
• MLP training is challenged by non-convex optimization landscapes with saddle points and local minima, leading to gradient vanishing and premature convergence.
• Existing heuristic algorithms like GA, GWO, and DE, and the Besiege and Conquer Algorithm (BCA), have limitations in search flexibility, detection, adaptation, and convergence speed.

Purpose of the Study:

• To propose a Flexible Besiege and Conquer Algorithm (FBCA) to overcome the limitations of traditional optimization algorithms in training MLPs.
• To enhance the search flexibility and convergence capability of optimization algorithms for complex deep learning tasks.
• To demonstrate the superior performance of FBCA in benchmark function tests and MLP optimization problems.

Main Methods:

• Introduced three novel mechanisms: sine-guided soft asymmetric Gaussian perturbation for enhanced local exploration.
• Implemented an exponentially modulated spiral perturbation for fast global convergence adaptation.
• Utilized a nonlinear cognitive coefficient-driven velocity update for balanced exploration-exploitation and improved convergence.

Main Results:

• FBCA achieved first place in the IEEE CEC 2017 benchmark function test against 12 state-of-the-art algorithms.
• FBCA demonstrated a 62% win rate over BCA in 100-dimensional problems.
• FBCA achieved superior performance in six MLP optimization problems, showing excellent convergence accuracy and robustness.

Conclusions:

• FBCA significantly improves search flexibility and convergence capability for complex nonlinear optimization problems.
• The proposed algorithm exhibits excellent global optimization ability, particularly in training Multi-Layer Perceptrons.
• FBCA shows strong application value and potential for optimizing neural networks and deep learning models.