一个循环神经回路的无条件稳定性,实现分裂性正常化
Shivang Rawat1,2, David J Heeger3,4, Stefano Martiniani1,2,5
1Courant Institute of Mathematical Sciences, NYU.
Advances in neural information processing systems
|November 7, 2025
概括
我们介绍了振荡反复门神经集成电路 (ORGaNICs),这是一个生物学上可信的模型,证明了无条件的稳定性. 这一突破允许在没有梯度问题的情况下进行训练,增强神经动力学模型的开发.
科学领域:
- 计算神经科学是一种神经科学.
- 机器学习 机器学习
- 动态系统 动态系统
背景情况:
- 经常出现的神经模型面临稳定性挑战,阻碍了生物学上可信和可训练的神经动力系统.
- 由于复杂的非线性,传统的皮层模型很难训练.
- 标准循环神经网络 (RNN) 缺乏生物可信性和解释性.
研究的目的:
- 将动态分裂规范化 (DN) 与振荡性反复门神经集成电路 (ORGaNICs) 的稳定性联系起来.
- 建立ORGaNICs作为一个生物可信的,稳定的,可训练的循环皮质电路模型.
- 为电路和神经元功能提供规范性原则.
主要方法:
- 利用Lyapunov的间接方法来证明任意维的ORGaNICs与同一重量矩阵的无条件局部稳定性.
- 连接ORGaNICs与合的减压波器来导出能量函数.
- 证明了2DORGaNICs模型的稳定性,具有通用反复重量矩阵和经验验证的更高维度.
主要成果:
- 对于具有相同重复重量矩阵的ORGaNICs,证明了无条件的局部稳定性.
- 导出了ORGaNICs的能量函数,提供了一个规范原则.
- 在更高维度的通用重量矩阵中证明了经验稳定性.
- 展示了ORGaNICs通过反向传播通过时间的成功训练,而无需梯度剪切/缩放.
结论:
- 有机体具有内在的稳定性,可以在没有常见的梯度问题 (爆炸,消失,振荡) 的情况下进行训练.
- 在静态图像分类中,ORGaNICs的表现优于其他神经动力学模型,并在顺序任务中与LSTM相匹配.
- 该模型为开发生物可信和有效训练的神经网络提供了一个有希望的方向.
相关概念视频
01:25Neural Circuits
2.6K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
2.6K
01:30Current Growth And Decay In RL Circuits
4.5K
The current growth and decay in RL circuits can be understood by considering a series RL circuit consisting of a resistor, an inductor, a constant source of emf, and two switches. When the first switch is closed, the circuit is equivalent to a single-loop circuit consisting of a resistor and an inductor connected to a source of emf. In this case, the source of emf produces a current in the circuit. If there were no self-inductance in the circuit, the current would rise immediately to a steady...
4.5K
01:37Neural Regulation
43.1K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
43.1K
01:24BIBO stability of continuous and discrete -time systems
873
System stability is a fundamental concept in signal processing, often assessed using convolution. For a system to be considered bounded-input bounded-output (BIBO) stable, any bounded input signal must produce a bounded output signal. A bounded input signal is one where the modulus does not exceed a certain constant at any point in time.
To determine the BIBO stability, the convolution integral is utilized when a bounded continuous-time input is applied to a Linear Time-Invariant (LTI) system....
To determine the BIBO stability, the convolution integral is utilized when a bounded continuous-time input is applied to a Linear Time-Invariant (LTI) system....
873
01:24Comparison between RL and RC circuits
5.9K
An RC circuit consists of resistance and capacitance, while in an RL circuit, capacitance is replaced by an inductor. RL and RC circuits are first-order differential circuits that store energy. An RC circuit stores energy in the electric field, while an RL circuit stores energy in the magnetic field. When connected to a battery, an RC circuit charges the capacitor, causing the current to decrease from maximum to zero upon being fully charged. This increases the voltage across the capacitor from...
5.9K
01:25Network Function of a Circuit
613
Frequency response analysis in electrical circuits provides vital insights into a circuit's behavior as the frequency of the input signal changes. The transfer function, a mathematical tool, is instrumental in understanding this behavior. It defines the relationship between phasor output and input and comes in four types: voltage gain, current gain, transfer impedance, and transfer admittance. The critical components of the transfer function are the poles and zeros.
613