A novel backpropagation algorithm based on negated kurtosis loss for training shallow, convolutional, and deep neural...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Backpropagation (BP) is the standard for training neural networks (NNs) like SNN, CNN, DNN, and DCNN.
• BP's Mean Square Error (MSE) loss function leads to slow convergence and high steady-state error.
• Existing methods struggle with optimizing NN training efficiency.

Purpose of the Study:

• To propose a novel BP algorithm for enhanced training and testing of SNN, CNN, DNN, and DCNN.
• To improve convergence rates and reduce steady-state errors in NN training.
• To introduce a new loss function based on negated output error kurtosis.

Main Methods:

• Developed a novel BP algorithm minimizing the negated kurtosis of the output layer's error.
• Extended the kurtosis-based BP algorithm to incorporate optimizers like RMSProp and Adam.
• Evaluated the algorithm on regression and classification tasks across various NN architectures.

Main Results:

• The proposed kurtosis-based BP algorithm demonstrated increased convergence rates.
• The novel BP algorithm significantly decreased steady-state errors compared to conventional methods.
• Improvements were observed across all tested NN architectures (SNN, CNN, DNN, DCNN).

Conclusions:

• The kurtosis-based BP algorithm offers superior performance over traditional BP.
• This novel approach enhances both training and testing efficiencies for diverse neural networks.
• Minimizing output error kurtosis is an effective strategy for NN optimization.