在非·诺伊曼架构中的基于尖端的Q学习
Donghyuk Shin1,2, Hyeongcheol Jo1,2, Hyeseung Jang1,2
1Korea University, Seoul, Republic of Korea.
Frontiers in neuroscience
|March 9, 2026
概括
本研究介绍了一种新的非·诺伊曼架构,使用尖端神经网络 (SNN) 进行高效的强化学习 (RL). 硬件可行的设计通过整合内存和计算来加速Q学习,降低功耗.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 神经科学是一个神经科学.
背景情况:
- ·诺伊曼架构面临数据传输瓶和高功耗由于内存计算分离.
- 强化学习 (RL) 工作负载,特别是Q学习,需要在大型国家行动空间中频繁更新,加剧这些问题.
- 尖端神经网络 (SNN) 通过事件驱动的稀疏处理提供计算效率.
研究的目的:
- 提出基于SNN的硬件可行的非·诺曼架构,以实现高效的Q学习.
- 克服RL任务的传统架构的局限性.
- 为了利用SNN的稀疏处理来提高计算效率.
主要方法:
- 开发了一个非·诺伊曼架构,将状态/动作映射到神经元和Q值映射到突触.
- 实现了横向抑制结构,用于识别更新的最大Q值.
- 整合了一个延迟电路,用于时间一致性和局部学习信号,用于针对性的突触更新.
主要成果:
- 在Cart-pole基准指标上的模拟显示了稳定的学习表现.
- 该架构实现了与基于软件的Q-learning具有足够比特精度的可比精度.
- 即使在低位精度条件下,也观察到有效的学习.
结论:
- 提出的基于SNN的非·诺伊曼架构有效地执行Q学习.
- 这种方法显著减少了数据传输瓶和RL工作负载的功耗.
- 该架构显示了高效,硬件加速的强化学习应用程序的前景.
相关概念视频
Observational Learning
1.1K
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
1.1K
Reinforcement Schedules
652
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
652
Current Growth And Decay In RL Circuits
4.8K
The current growth and decay in RL circuits can be understood by considering a series RL circuit consisting of a resistor, an inductor, a constant source of emf, and two switches. When the first switch is closed, the circuit is equivalent to a single-loop circuit consisting of a resistor and an inductor connected to a source of emf. In this case, the source of emf produces a current in the circuit. If there were no self-inductance in the circuit, the current would rise immediately to a steady...
4.8K
Multi-input and Multi-variable systems
453
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
453
Associative Learning
1.7K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.7K
Woodward–Hoffmann Selection Rules and Microscopic Reversibility
4.1K
Electrocyclic reactions, cycloadditions, and sigmatropic rearrangements are concerted pericyclic reactions that proceed via a cyclic transition state. These reactions are stereospecific and regioselective. The stereochemistry of the products depends on the symmetry characteristics of the interacting orbitals and the reaction conditions. Accordingly, pericyclic reactions are classified as either symmetry-allowed or symmetry-forbidden. Woodward and Hoffmann presented the selection criteria for...
4.1K


