Related Experiment Videos
Complexity issues in natural gradient descent method for training multilayer perceptrons
1Oregon Graduate Institute, Computer Science Dept, Box 91000, Portland OR 97291, USA. hyang@cse.ogi.edu
Neural Computation
|November 6, 1998
Abstract:
The natural gradient descent method is applied to train an n-m-1 multilayer perceptron. Based on an efficient scheme to represent the Fisher information matrix for an n-m-1 stochastic multilayer perceptron, a new algorithm is proposed to calculate the natural gradient without inverting the Fisher information matrix explicitly. When the input dimension n is much larger than the number of hidden neurons m, the time complexity of computing the natural gradient is O(n).