在歌鸟获取强化学习模型中的重量转移
Khue Tran1,2, Alexei Koulakov1
1Cold Spring Harbor Laboratory, Cold Spring Harbor, NY, USA.
bioRxiv : the preprint server for biology
|January 13, 2025
概括
这项研究模拟了歌鸟的学习,这个过程类似于人类的语音学习. 它整合了强化学习和Hebbian学习,以解释鸟类如何通过试错来学习歌曲.
科学领域:
- 神经科学是一个神经科学.
- 计算生物学 计算生物学
- 动物行为 动物行为
背景情况:
- 歌鸟的声学学习提供了对人类语言获取的洞察力.
- 试错学习对于获得复杂的发音至关重要.
- 现有的模型不能完全捕捉歌曲学习的神经电路.
研究的目的:
- 介绍歌鸟学习歌曲的计算模型.
- 将强化学习 (RL) 和Hebbian学习整合到一个生物学上可信的框架内.
- 阐明声学学习背后的神经机制及其与人类语言的相似之处.
主要方法:
- 开发了一个模拟歌鸟神经电路 (HVC,RA,LMAN,X区,VTA) 的计算模型.
- 通过HVC-to-Area X路径使用强化学习进行初始歌曲获取.
- 利用Hebbian学习和尖峰时间依赖的可塑性 (STDP) 进行歌曲整合和通过丘脑传输.
主要成果:
- 该模型通过两阶段的学习过程成功模拟了歌曲的获取.
- 在HVC-to-Area X路径中的强化学习指导了最初的政策制定.
- 赫比安可塑性和STDP巩固学习并将其转移到HVC-to-RA路径.
结论:
- 拟议的模型为理解歌鸟的声学学习提供了一个统一的框架.
- 整合RL和Hebbian学习解释了歌曲获取的顺序性质.
- 这种计算方法为研究学习和语言的神经基础提供了有价值的工具.
相关概念视频
Instinctive Drift
181
Instinctive drift refers to the tendency of animals to revert to their innate behaviors despite repeated reinforcement. Breland and Breland demonstrated this concept in an experiment with a raccoon. The raccoon was trained to pick up two coins and place them in a container in exchange for food. Initially, the raccoon learned to associate the coins with food, making them a conditioned stimulus or a substitute for food. However, over time, the raccoon became less willing to put the coins into the...
181
Associative Learning
286
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
286
Reinforcement Schedules
130
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
130
State Space to Transfer Function
171
The conversion of state-space representation to a transfer function is a fundamental process in system analysis. It provides a method for transitioning from a time-domain description to a frequency-domain representation, which is crucial for simplifying the analysis and design of control systems.
The transformation process begins with the state-space representation, characterized by the state equation and the output equation. These equations are typically represented as:
The transformation process begins with the state-space representation, characterized by the state equation and the output equation. These equations are typically represented as:
171
Transfer Function to State Space
192
State-space representation is a powerful tool for simulating physical systems on digital computers, necessitating the conversion of the transfer function into state-space form. Consider an nth-order linear differential equation with constant coefficients, like those encountered in an RLC circuit. The state variables are selected as the output and its n−1 derivatives. Differentiating these variables and substituting them back into the original equation produces the state equations.
In an...
In an...
192
Apparent Weight
8.1K
True weight is the measure of the gravitational force acting on an object. However, if the object accelerates, its measured weight is different from its true weight. Similar observations can be made when the object is submerged in water. An object's weight in water is its apparent weight, which is equal to the difference between its true weight and the buoyant forces.
Consider a person standing on a bathroom scale inside an elevator. If the scale is accurate at rest, its reading equals the...
Consider a person standing on a bathroom scale inside an elevator. If the scale is accurate at rest, its reading equals the...
8.1K


