Multiobjective hybrid optimization and training of recurrent neural networks

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Optimizing neural networks, particularly recurrent neural networks (RNNs), is computationally intensive.
• Traditional methods involve trial-and-error for network topology selection and training, which can be inefficient.
• Recurrent neural networks (RNNs) often face challenges like vanishing gradient problems, hindering effective training.

Purpose of the Study:

• To develop and evaluate a method for simultaneous training and topology optimization of RNNs.
• To address the computational cost and training difficulties associated with RNNs.
• To investigate the impact of hybridization strategies and evolutionary algorithm parameters on optimization diversity.

Main Methods:

• Utilized multiobjective hybrid procedures based on SPEA2 and NSGA2 algorithms.
• Employed the Baldwinian hybridization strategy for combining evolutionary and local search methods.
• Investigated the influence of objective selection, crossover, and mutation on evolutionary diversity.

Main Results:

• Successfully demonstrated simultaneous training and topology optimization for RNNs.
• The proposed hybrid approach effectively navigated the complex optimization landscape.
• Analysis provided insights into how parameter choices affect diversity in the evolutionary process.

Conclusions:

• The presented multiobjective hybrid approach offers an efficient solution for RNN training and topology optimization.
• This method mitigates common challenges in RNN development, such as computational cost and vanishing gradients.
• The findings contribute to advancing automated machine learning and neural network design.