在以昆虫体为灵感的间隙连接网络中获得稀少奖励的学习
Tianqi Wei1,2, Qinghai Guo3, Barbara Webb1
1Institute of Perception, Action, and Behaviour, School of Informatics, University of Edinburgh, Edinburgh, United Kingdom.
PLoS computational biology
|May 23, 2024
概括
这项研究引入了一种新的强化学习算法,灵感来自昆虫神经回路,以解决稀疏奖励问题. 该算法使用自适应连接来有效地学习具有延迟奖励的任务.
科学领域:
- 神经科学是一个神经科学.
- 机器学习 机器学习
- 计算生物学 计算生物学
背景情况:
- 动物面临的学习挑战与稀疏的奖励,加强延迟到任务完成.
- 传统的强化学习算法与这种"远程"或"稀疏"奖励问题作斗争.
- 昆虫体表现出并行纤维,具有潜在的axo-axonal间隙连接连接,形成电阻网络.
研究的目的:
- 在奖励稀疏的场景中研究一种用于强化学习的新型算法.
- 探索神经回路中轴轴间隙连接的潜力,作为决策模型.
- 通过自适应连接提出基于图形的方法来编码任务结构.
主要方法:
- 开发了一种强化学习算法,其灵感来源于由间隙连接形成的电阻网络.
- 建模的任务状态作为主动节点,状态过渡作为与自适应权重的连接.
- 模拟电流流向目标状态以指导图形结构中的决策.
主要成果:
- 在使用拟议的算法获得稀少奖励的任务中证明了高效的强化学习.
- 展示了适应性差距连接重量如何调节连接以编码任务结构.
- 该算法通过任务的图形表示有效指导决策.
结论:
- 拟议的算法提供了一个有效的解决方案,用于在稀疏的奖励下进行强化学习.
- 该模型为昆虫体的功能提供了可信的计算解释.
- 轴-轴突间隙结可能在生物系统的学习和决策中发挥关键作用.
相关概念视频
Neural Circuits
1.2K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.2K
Chemical Synapses
2.5K
Chemical synapses are specialized sites between two neurons or between a neuron and a non-neuronal cell like a muscle, glandular or sensory cell.
Because chemical synapses depend on the release of neurotransmitter molecules from synaptic vesicles to pass on their signal, there is an approximately one millisecond delay between when the axon potential reaches the presynaptic terminal and when the neurotransmitter leads to opening of postsynaptic ion channels. Additionally, this signaling is...
Because chemical synapses depend on the release of neurotransmitter molecules from synaptic vesicles to pass on their signal, there is an approximately one millisecond delay between when the axon potential reaches the presynaptic terminal and when the neurotransmitter leads to opening of postsynaptic ion channels. Additionally, this signaling is...
2.5K
Gap Junctions
8.0K
The cytoplasm of adjacent animal cells can exchange small molecules, ions, and secondary messengers via the communication channels which form the gap junctions. These junctions comprise a few hundred to thousands of molecular channels, each made of two halves, called the connexon hemichannel. A connexon is a hexamer of six transmembrane connexin proteins, which assemble radially, thus forming a pore or channel in the center. One connexon hemichannel docks with a corresponding connexon on the...
8.0K
Neurons: The Axon
3.4K
Axons are long, cytoplasmic processes of nerve cells capable of propagating electrical impulses known as action potentials. The cytoplasm or axoplasm of an axon contains neurofibrils, neurotubules, small vesicles, lysosomes, mitochondria, and various enzymes, all encased within the axolemma, the plasma membrane of the axon.
The axon attaches to the cell body at a cone-shaped elevation called the axon hillock. The initial part of the axon, closest to the hillock, is known as the initial segment....
The axon attaches to the cell body at a cone-shaped elevation called the axon hillock. The initial part of the axon, closest to the hillock, is known as the initial segment....
3.4K
Overview of Synapses
2.3K
A synapse is a specialized structure where two neurons connect, allowing them to pass an electrical or chemical signal to another neuron. It is the point of communication between neurons. The term "synapse" is derived from the Greek word "synapsis," which means "conjunction." The entire process of neural communication revolves around the synapse. When activated, a neuron releases chemicals known as neurotransmitters into the synapse. These neurotransmitters cross the synapse and bind to...
2.3K


