Multiplicative processing in the modeling of cognitive activities in large neural networks

Area of Science:

• Neuroscience
• Computational Neuroscience
• Biophysics

Background:

• The study of cognitive abilities in neural systems began with biophysics in the 1940s, linked to early computing and artificial intelligence.
• Neural network models have evolved significantly in both biological and computational fields.
• Distributed Associative Memory models in the 1970s highlighted the necessity of signal multiplication for adaptive associations.

Purpose of the Study:

• To review the role of signal multiplication in neural network models.
• To explore the computational reasons for including multiplication in these models.
• To discuss potential biophysical implementations and implications for deep learning.

Main Methods:

• Review of classical neural network models incorporating signal multiplication.
• Analysis of theoretical models proposing tensor product representations.
• Examination of deep-learning models and their reliance on multiplication.

Main Results:

• Many successful neural network models utilize signal multiplication for enhanced computational capabilities.
• Tensor product representations provide context-dependent adaptive memory capabilities.
• Modern deep-learning models also depend on multiplication for their powerful abilities.

Conclusions:

• Signal multiplication is a fundamental computational element in neural networks, from classical models to contemporary deep learning.
• Understanding the biophysical mechanisms of neural multiplication is essential for future research.
• This principle is key to evolutionary adaptation in unpredictable environments.