Bi-PIL: Bidirectional Gradient-Free Learning Scheme for Multilayer Neural Networks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Traditional deep neural network training relies on gradient descent, which is time-consuming.
• Designing complex neural network architectures is often intractable with current methods.

Purpose of the Study:

• To explore an efficient gradient-free learning scheme for building multilayer neural networks.
• To offer a potential solution for automated neural network architectural design.

Main Methods:

• The proposed scheme uses a bidirectional training (BT) process, encompassing forward and backward training (FT).
• Pseudoinverse learning (PIL) algorithm trains the network layer-by-layer in a greedy, data-driven manner during FT.
• Network architecture and weights from FT are used in the gradient-free backward process to update weights.

Main Results:

• A neural network with two twin subnetworks is obtained after bidirectional learning.
• Fused features from subnetworks serve as inputs for downstream tasks.
• Experiments demonstrate the effectiveness and superiority of the proposed learning scheme.

Conclusions:

• The proposed gradient-free learning scheme efficiently constructs multilayer neural networks.
• This approach simplifies architectural design and accelerates training compared to gradient descent.
• The resulting twin subnetworks provide fused features beneficial for various downstream applications.