Deterministic Convergence Analysis and Application of Elman Neural Network via Sparse Mechanism and Entropy Error...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neural Networks

Background:

• Traditional mean square error functions in Elman neural networks (ENN) can lead to slow convergence and local minima.
• Existing regularization methods may cause oscillations in the error function during ENN training.

Purpose of the Study:

• To develop a novel entropy error function (EEF) for ENN training to enhance learning speed and avoid degradation.
• To apply smoothing group L1/2 regularization (SGL1/2) to address oscillations in ENN training.
• To optimize ENN architecture for improved sparsity and generalization.

Main Methods:

• Employed batch gradient method to analyze Elman neural network (ENN) monotonicity and convergence.
• Introduced a novel entropy error function (EEF) for ENN training.
• Utilized smoothing group L1/2 regularization (SGL1/2) to stabilize the error function.
• Optimized network architecture by reducing nodes and weights to enhance sparsity.

Main Results:

• The novel EEF effectively avoids learning speed degradation issues.
• SGL1/2 regularization overcomes oscillations associated with traditional group L1/2 regularization (GL1/2).
• Network optimization significantly boosts sparsity by minimizing redundant nodes and weights.
• Theoretical proofs confirm the monotonicity and convergence (strong and weak) of the proposed method.

Conclusions:

• The proposed method using EEF and SGL1/2 regularization enhances ENN stability, sparsity, and generalization.
• Experimental results validate the theoretical findings, demonstrating the effectiveness of the approach.
• This work offers a robust alternative for training ENNs, addressing key limitations of existing techniques.