量化和最大化信息流在循环神经网络中的信息流
Claus Metzner1,2, Marius E Yamakou3, Dennis Voelkl4
1Neuroscience Lab, University Hospital Erlangen, 91054 Erlangen, Germany.
Neural computation
|February 16, 2024
概括
研究人员开发了一种方法来量化和最大化循环神经网络 (RNN) 中的信息流. 这种方法在大型系统中使用神经元对之间的相关性,有助于设计RNN用于记忆和模式生成.
科学领域:
- 计算神经科学是一种神经科学.
- 人工智能的人工智能
- 信息理论 信息理论
背景情况:
- 循环神经网络 (RNN),特别是概率模型,产生连续的信息流.
- 信息流量是通过连续网络状态之间的相互信息来量化.
- 以前的工作将信息流与连接重量统计联系起来,但缺乏系统最大化和大规模量化方法.
研究的目的:
- 系统地最大化RNN中的信息流.
- 开发用于量化大规模系统中的信息流量的方法.
- 探索具有高自发信息流量的RNNs的设计原则.
主要方法:
- 使用博尔茨曼机器作为分析模型系统.
- 使用相互信息的量化信息流 I[x→(t),x→(t+1) ].
- 采用进化算法来最大限度地提高信息流量和循环吸引器周期长度.
主要成果:
- 相互信息 I 是中度连接网络中根-平均-平方平均的皮尔森相关的单调转换.
- 皮尔森相关性提供了一种有效的方法来量化大型系统中的信息流.
- 进化最大化确定了重量矩阵的设计原则,以增强自发的信息流.
结论:
- 建立了一种有效的方法来量化和最大化RNN中的信息流,适用于大型系统.
- 发现了用于构建具有高自发信息流量的RNN的设计原则.
- 证明了信息流量和吸引器周期长度的同时最大化,对短期记忆和模式生成应用有用.
相关概念视频
Neural Circuits
1.2K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.2K
The Role of Ion Channels in Neuronal Computation
3.2K
A postsynaptic neuron usually receives numerous impulses from several other presynaptic neurons. The axon hillock of the postsynaptic neuron integrates all these signals and determines the likelihood of firing an action potential.
Sometimes a single EPSP is strong enough to induce an action potential in the postsynaptic neuron. However, multiple presynaptic inputs must often create EPSPs around the same time for the postsynaptic neuron to be sufficiently depolarized to fire an action potential....
Sometimes a single EPSP is strong enough to induce an action potential in the postsynaptic neuron. However, multiple presynaptic inputs must often create EPSPs around the same time for the postsynaptic neuron to be sufficiently depolarized to fire an action potential....
3.2K
Integration of Synaptic Events
1.5K
Synaptic integration mainly includes the summation of graded potentials. Graded potentials, regardless of their type, cause subtle alterations in membrane voltage, resulting in either depolarization or hyperpolarization. These incremental changes, when combined or summed, can propel the neuron toward its threshold. Consider, for example, a membrane experiencing a +15 mV shift, causing it to depolarize from -70 mV to -55 mV. In this scenario, graded potentials govern the membrane's ability to...
1.5K
Neuroplasticity
359
Neuroplasticity reflects the brain's remarkable capacity to adapt and evolve, responding dynamically to learning, experiences, or injury by reorganizing its neural circuitry. This reorganization involves creating new neural connections and refining old ones through a series of biological processes that contribute to the brain's lifelong development and adaptability.
359


