Related Experiment Videos
Stabilization of recurrent neural networks through divisive normalization
Flaviano Morone1,2, Shivang Rawat2,3, David J Heeger1,4
1Center for Neural Science, New York University, New York, NY 10003.
Summary
Divisive normalization enhances neural circuit stability by allowing recurrent neural networks to remain stable despite large synaptic weights. This mechanism is crucial for complex computations in the brain.
Area of Science:
- Computational neuroscience
- Dynamical systems theory
- Neural circuit dynamics
Background:
- Neural circuit stability is essential but challenging with recurrent connections.
- Linear models require precise synaptic weights for stability, which is biologically unrealistic.
- Recurrent interactions are vital for complex cognitive functions.
Purpose of the Study:
- To investigate how divisive normalization affects the stability of recurrent neural networks.
- To determine if normalization can maintain stability beyond the limits of standard linear models.
- To explore the relationship between normalization, synaptic strength, and critical slowing down.
Main Methods:
- Theoretical analysis of dynamical systems.
- Numerical simulations of recurrent neural networks with divisive normalization.
- Analytical prediction of normalization breakdown and critical slowing down.
Main Results:
- Recurrent neural networks with divisive normalization can remain stable even with spectral radii exceeding 1.
- Instability is preceded by critical slowing down, an early warning signal.
- The onset of critical slowing down correlates with the breakdown of normalization.
Conclusions:
- Divisive normalization is a key mechanism for ensuring neural circuit stability.
- This stability is achieved despite strong recurrent interactions necessary for complex computations.
- The findings highlight normalization's role in biological and engineered neural systems.
Related Concept Videos
01:37Neural Regulation
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
01:19Survival Tree
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a survival tree begins...
Building a Survival Tree
Constructing a survival tree begins...
01:52Regression Toward the Mean
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when researchers try to extrapolate results...