Related Experiment Videos
Convergence analysis of three classes of split-complex gradient algorithms for complex-valued recurrent neural
Dongpo Xu1, Huisheng Zhang, Lijun Liu
1College of Science, Harbin Engineering University, Harbin, People's Republic of China. dongpoxu@gmail.com
Abstract:
This letter presents a unified convergence analysis of the split-complex nonlinear gradient descent (SCNGD) learning algorithms for complex-valued recurrent neural networks, covering three classes of SCNGD algorithms: standard SCNGD, normalized SCNGD, and adaptive normalized SCNGD. We prove that if the activation functions are of split-complex type and some conditions are satisfied, the error function is monotonically decreasing during the training iteration process, and the gradients of the error function with respect to the real and imaginary parts of the weights converge to zero. A strong convergence result is also obtained under the assumption that the error function has only a finite number of stationary points. The simulation results are given to support the theoretical analysis.
Related Concept Videos
Gradient Vectors and Their Applications
Significance of the Gradient Vector
Region of Convergence of Laplace Tarnsform
Consider a decaying exponential signal that begins at a specific time. When deriving its Laplace transform, the time-domain variable is replaced with a complex variable. This substitution...
Gradient Fields
Convergence of Sequences
Gradient and Del Operator