The Whale Optimization Algorithm Approach for Deep Neural Networks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep learning faces significant challenges in parameter selection and optimization.
• Existing methods struggle with issues like local minima, saddle points, and vanishing gradients.
• Bio-inspired algorithms offer potential solutions but require further development.

Purpose of the Study:

• To introduce the Whale Optimization Algorithm (WOA) for optimizing neural network hyperparameters.
• To address the limitations of current hyperparameter optimization techniques.
• To present the first known application of WOA for hyperparameter optimization in deep learning.

Main Methods:

• Detailed description and formulation of the Whale Optimization Algorithm (WOA).
• Application of WOA for hyperparameter optimization in deep learning models.
• Implementation and comparison with Grid Search, Random Search, and a 3D-WOA variant.
• Utilizing the swarm foraging behavior of humpback whales for optimization.

Main Results:

• The proposed WOA algorithm successfully optimizes neural network hyperparameters.
• Achieved high accuracy rates: 89.85% on Fashion MNIST and 80.60% on Reuters datasets.
• Demonstrated competitive performance against established optimization methods.

Conclusions:

• The Whale Optimization Algorithm is a viable and effective method for deep learning hyperparameter optimization.
• WOA offers a promising bio-inspired alternative to traditional optimization techniques.
• Further research can explore extensions like the 3D-WOA for enhanced performance.