Nonlinear complex-valued extensions of Hebbian learning: an essay

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• The Hebbian paradigm is a foundational unsupervised learning theory in connectionism, known for its locality and applicability to neuron models.
• Despite its strengths, the basic Hebbian principle has theoretical limitations hindering practical application in signal and data processing.
• Modifications over two decades have led to principal component analysis (PCA) type learning rules and nonlinear extensions.

Purpose of the Study:

• To present a unified view of fragmented material in principal component learning.
• To motivate and present extensions of Hebbian learning to complex-weighted linear neural networks.
• To analyze complex-valued principal/minor component/subspace linear/nonlinear rules for complex-weighted neural structures.

Main Methods:

• Reviewing and synthesizing existing literature on Hebbian learning modifications and PCA-type rules.
• Extending previous studies on linear signal decomposition and nonquadratic component optimization.
• Applying constrained optimization for neural parameter adaptation with complex-valued arguments.
• Incorporating topological elements or modifying learning criteria for orthonormality.

Main Results:

• A unified framework for principal component learning rules is presented.
• Extensions of Hebbian learning to complex-weighted linear neural networks are proposed.
• Analysis covers linear and nonlinear rules for complex-valued principal/minor component/subspace learning.
• Consideration of both feedforward and laterally connected complex-weighted neural structures.

Conclusions:

• This work provides a cohesive perspective on principal component learning derived from Hebbian principles.
• The proposed extensions enhance the applicability of Hebbian learning to complex-valued neural networks.
• The findings contribute to the development of advanced signal and data processing algorithms.