Related Experiment Videos
A note on Lewicki-Sejnowski gradient for learning overcomplete representations
Zhaoshui He1, Shengli Xie, Liqing Zhang
1School of Electronics and Information Engineering, South China University of Technology, Guangzhou, 510640, China. zhshhe@scut.edu.cn
Abstract:
Overcomplete representations have greater robustness in noise environment and also have greater flexibility in matching structure in the data. Lewicki and Sejnowski (2000) proposed an efficient extended natural gradient for learning the overcomplete basis and developed an overcomplete representation approach. However, they derived their gradient by many approximations, and their proof is very complicated. To give a stronger theoretical basis, we provide a brief and more rigorous mathematical proof for this gradient in this note. In addition, we propose a more robust constrained Lewicki-Sejnowski gradient.
Related Concept Videos
Significance of the Gradient Vector
Gradient Vectors and Their Applications
Gradient Fields
Gradient and Del Operator
Introduction to Learning
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
Vector Algebra: Method of Components
In many applications, the magnitudes and directions of...