Complete Stability of Delayed Recurrent Neural Networks With New Wave-Type Activation Functions.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Dynamical Systems

Background:

• Activation functions critically influence neural network dynamics.
• Delayed recurrent neural networks (DRNNs) are essential for modeling complex temporal processes.
• Existing activation functions may limit the memory capacity of DRNNs.

Purpose of the Study:

• To propose novel nonmonotonic wave-type activation functions.
• To analyze the complete stability of DRNNs utilizing these new activation functions.
• To enhance the memory storage capacity of DRNNs.

Main Methods:

• Utilizing geometrical properties of the proposed wave-type activation functions.
• Employing a subsequent iteration scheme for stability analysis.
• Deriving sufficient conditions for the number and stability of equilibria in DRNNs.

Main Results:

• DRNNs with proposed activation functions achieve exactly $(2m + 3)^{n}$ equilibria.
• $(m + 2)^{n}$ equilibria are locally exponentially stable, while the rest are unstable.
• The proposed activation functions ensure complete stability and significantly increase memory storage capacity compared to prior work.

Conclusions:

• The novel wave-type activation functions offer superior performance in DRNNs.
• These functions lead to a substantial increase in both total and stable equilibria.
• The enhanced stability and memory capacity have significant implications for advanced neural network applications.