BatTS: a hybrid method for optimizing deep feedforward neural network.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep feedforward neural networks (DFNNs) demonstrate significant success across computational tasks.
• Current DFNN architecture selection relies on inefficient manual or trial-and-error approaches.
• Automating DFNN architecture design is crucial for achieving state-of-the-art performance but remains laborious.

Purpose of the Study:

• To introduce a novel hybrid methodology, BatTS, for optimizing deep feedforward neural network architectures.
• To enhance the efficiency and effectiveness of DFNN architecture design processes.
• To improve the overall performance of DFNNs through automated architecture optimization.

Main Methods:

• A hybrid methodology (BatTS) integrating the Bat algorithm, Tabu search (TS), and Gradient Descent with Momentum (GDM) backpropagation.
• Dynamic architecture generation guided by the Bat algorithm.
• Efficient architecture evaluation and local optima escape facilitated by Tabu search.
• Empirical evaluation on four diverse benchmark datasets.

Main Results:

• BatTS demonstrated improved performance compared to traditional Tabu search and random trial methods.
• The hybrid approach showed enhanced efficiency in exploring and evaluating new architectures.
• The dynamic nature of BatTS aids in discovering superior DFNN architectures.
• Consistent performance gains observed across multiple benchmark datasets.

Conclusions:

• The proposed BatTS hybrid methodology offers a more effective and efficient approach to optimizing DFNN architectures.
• BatTS overcomes limitations of manual design and random search by combining metaheuristic optimization with robust training.
• This work contributes a valuable tool for researchers and practitioners seeking to advance DFNN performance through intelligent architecture design.