在重尾循环神经网络中,缓慢过渡到低维混乱

Eva Yi Xie^1,2, Stefan Mihalas¹, Łukasz Kuśmierz¹

¹Allen Institute, Seattle, WA, USA.

ArXiv

|November 24, 2025

概括

与高斯网络不同,具有重尾突触权重的循环神经网络 (RNN) 显示了强大的混乱过渡. 这种生物现实主义提供了动态稳定性和神经活动丰富性之间的权衡.

科学领域:

计算神经科学是一种计算神经科学.
理论神经科学理论神经科学
机器学习机器学习

背景情况:

大脑中的突触重量通常遵循重尾分布.
经常性神经网络 (RNN) 的理论分析通常假定高斯连接,这可能不是生物学上准确的.

研究的目的:

为了研究RNN的动态,用从重尾莱维α稳定分布中提取的突触重量来研究RNN的动态.
分析有限大小的重尾RNN中静止和混乱状态之间的过渡.
了解生物现实的神经网络中强度和维度之间的权衡.

主要方法:

用莱维α稳定的分布随机权重对RNN活性进行系统研究.
有限大小分析以确定从静止到混乱动态的过渡点.
理论预测和基于模拟的验证过渡收益.
分析吸引子的Lyapunov维度,以评估有效的维度.

主要成果:

有限重尾RNN表现出静止和混乱动态之间的急剧过渡,与无处不在的混乱的平均场预测形成鲜明对比.
在重尾RNN中观察到接近混乱边缘的更广泛的增强模式,这表明向混乱的过渡速度更慢.
在突触重量中较重的尾巴会导致减少Lyapunov维度,这意味着神经活动的有效维度较低.

结论:

在RNN中,重尾连接提供了一个生物学上可信的模型,具有强大的混乱过渡.
在混乱边缘附近的动态的稳定性和神经活动的丰富性 (维度) 之间存在一个权衡.
这项研究提供了一个可操作的框架,通过描述有限大小网络中的过渡点来分析现实大小的重尾神经电路中的动态.

相关概念视频

01:19

Survival Tree

374

Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...

374

01:10

Entropy Change in Reversible Processes

3.2K

In the Carnot engine, which achieves the maximum efficiency between two reservoirs of fixed temperatures, the total change in entropy is zero. The observation can be generalized by considering any reversible cyclic process consisting of many Carnot cycles. Thus, it can be stated that the total entropy change of any ideal reversible cycle is zero.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.

3.2K

01:24

Sequence Networks of Rotating Machines

473

A Y-connected synchronous generator, grounded through a neutral impedance, is designed to produce balanced internal phase voltages with only positive-sequence components. The generator's sequence networks include a source voltage that is exclusively in the positive-sequence network. The sequence components of line-to-ground voltages at the generator terminals illustrate this configuration.
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...

473

01:09

Random Variables

17.2K

A random variable is a single numerical value that indicates the outcome of a procedure. The concept of random variables is fundamental to the probability theory and was introduced by a Russian mathematician, Pafnuty Chebyshev, in the mid-nineteenth century.
Uppercase letters such as X or Y denote a random variable. Lowercase letters like x or y denote the value of a random variable. If X is a random variable, then X is written in words, and x is given as a number.
For example, let X = the...

17.2K

01:25

Neural Circuits

2.6K

Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...

2.6K

00:59

Propagation of Uncertainty from Random Error

1.6K

An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...

1.6K