深度强化学习与显著的乘法推理推理
Dmitry A Ivanov1,2, Denis A Larionov2,3, Mikhail V Kiselev2,3
1Lomonosov Moscow State University, GSP-1, Leninskie Gory, Moscow, 119991, Russia.
Scientific reports
|November 27, 2023
概括
我们开发了一种用于强化学习 (RL) 中神经网络的稀疏计算方法. 这种方法显著减少了用于更快的神经形态计算的计算,而性能影响最小.
科学领域:
- 计算神经科学是一种神经科学.
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 神经网络推断是计算密集的,限制了资源有限的环境中的应用.
- 强化学习 (RL) 任务通常需要复杂的神经网络模型.
- 神经形态计算旨在实现高效,以大脑为灵感的计算.
研究的目的:
- 提出一种新的稀疏计算方法,以优化RL中的神经网络推理.
- 为了减少计算负载,特别是乘法次数,以实现更快的处理.
- 为了提高计算效率,利用大脑启发的机制.
主要方法:
- 结合神经网络修剪 (模仿神经可塑性) 使用三角形网络算法.
- 神经网络修剪消除了多余的连接.
- 只有当变化超过一个值时,Delta网络才会更新神经元状态,利用输入数据的相关性.
主要成果:
- 在神经网络推理过程中实现了多达100倍的乘法减少.
- 在受欢迎的深度RL任务中保持了性能水平.
- 在某些情况下观察到性能改善.
结论:
- 拟议的稀疏计算方法为RL中的神经网络推理提供了显著的效率增长.
- 这种由大脑启发的方法适用于快速的神经形态计算.
- 该方法有效地降低了计算成本,而不会影响,有时甚至提高任务性能.
相关概念视频
Long-term Potentiation
55.3K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
55.3K
Convolution: Math, Graphics, and Discrete Signals
264
In any LTI (Linear Time-Invariant) system, the convolution of two signals is denoted using a convolution operator, assuming all initial conditions are zero. The convolution integral can be divided into two parts: the zero-input or natural response and the zero-state or forced response, with t0 indicating the initial time.
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
To simplify the convolution integral, it is assumed that both the input signal and impulse response are zero for negative time values. The graphical convolution process...
264
Cognitive Learning
246
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
246
Elaborative Rehearsals
89
Elaborative rehearsal is a crucial cognitive strategy that strengthens information encoding in long-term memory by making meaningful connections between new data and pre-existing knowledge. This approach contrasts with maintenance rehearsal, which involves simple repetition without delving into the significance of the information. While maintenance rehearsal might temporarily keep information active in short-term memory, it is less effective for long-term retention.
The effectiveness of...
The effectiveness of...
89
Real-World Application of Classical Conditioning
596
Classical conditioning not only includes the initial pairing of stimuli but also extends to more complex forms, such as higher-order conditioning. Higher-order conditioning involves creating associations beyond the primary conditioned stimulus, resulting in a chain of conditioned responses.
Higher-order, or second-order, conditioning occurs when a neutral stimulus becomes associated with an already established conditioned stimulus through repeated pairings. For instance, if a dog has been...
Higher-order, or second-order, conditioning occurs when a neutral stimulus becomes associated with an already established conditioned stimulus through repeated pairings. For instance, if a dog has been...
596
Convolution Properties I
153
Convolution computations can be simplified by utilizing their inherent properties.
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
153


