Evolving Deep Neural Networks via Cooperative Coevolution With Backpropagation.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep neural networks (DNNs) achieve success via complex architectures and feature learning.
• Gradient-based backpropagation (BP) is standard for DNN parameter learning but suffers from initialization sensitivity and local optima.
• Existing methods struggle with efficient and effective DNN training.

Purpose of the Study:

• To propose a novel DNN learning framework, BPCC, that hybridizes cooperative co-evolution (CC) with BP.
• To develop a computationally efficient CC-based optimization technique for DNN parameter learning.
• To enhance DNN training by mitigating issues associated with traditional BP methods.

Main Methods:

• The BPCC framework intermittently uses BP for training epochs.
• Cooperative co-evolution (CC) is activated when BP fails to sufficiently decrease the objective function.
• Parameter learning is decomposed into subtasks (e.g., neuron-based) addressed cooperatively, with a maturity-based strategy for subtask selection.

Main Results:

• Experimental results show the proposed BPCC method outperforms common DNN parameter learning techniques.
• The hybrid approach effectively addresses initialization sensitivity and local optima.
• The CC-based optimization with a maturity-based selection strategy proves computationally efficient.

Conclusions:

• BPCC offers a superior alternative to conventional DNN training methods.
• The hybridization of CC and BP enhances the robustness and efficiency of DNN parameter learning.
• This framework provides a promising direction for advancing deep learning optimization.