带有多分部神经元的尖端世界模型用于基于模型的强化学习
Yinqian Sun1,2, Feifei Zhao1,2,3, Mingyang Lyu1,4
1Brain-inspired Cognitive AI Lab, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China.
概括
这项研究介绍了一种用于强化学习的新型尖端世界模型 (Spiking-WM). 带有增强时间记忆的尖端神经网络 (SNN) 现在在决策任务中实现了与传统模型相比的性能.
科学领域:
- 计算神经科学是一种神经科学.
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 尖端神经网络 (SNN) 在感知方面表现有前途,但在基于模型的强化学习 (RL) 中未得到充分探索.
- 现有的SNN在RL中准确的世界模型所需的长期时间记忆中扎.
- 生物神经元的动态树突融合为改善SNN能力提供了洞察力.
研究的目的:
- 开发一种新的多分部神经元模型,用于SNN中增强时间信息处理.
- 使用SNN构建一个基于模型的深度强化学习的尖端世界模型 (Spiking-WM).
- 评估Spiking-WM的性能和长期记忆能力.
主要方法:
- 提出了一种用于非线性,多状信息集成的多分部神经元模型.
- 开发了Spiking-WM,集成了尖端状态空间,卷积和政策网络.
- 在DeepMind控制套件和多个语音数据集上评估了Spiking-WM.
主要成果:
- 在RL任务中,Spiking-WM在现有SNN模型中表现出优越的性能.
- 实现了与使用Gated Recurrent Units (GRUs) 的人工神经网络 (ANN) 世界模型相美的性能.
- 多分部神经元模型在处理长长的语音序列方面超过了其他SNN.
结论:
- 拟议的多分部神经元模型增强了SNN对RL的时间处理.
- 使用SNN,Spiking-WM有效地实现了基于模型的深度强化学习.
- 这种方法促进了SNN的复杂决策和长序数据处理.
相关概念视频
Multicompartment Models: Overview
474
Multicompartment models are mathematical constructs that depict how drugs are distributed and eliminated within the body. They segment the body into several compartments, symbolizing various physiological or anatomical areas connected through drug transfer processes such as absorption, metabolism, distribution, and elimination.
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
474
Compartment Models: Single-Compartment Model
3.0K
The single-compartment model serves as a simplified representation of the human body. This model assumes that the body functions as a single, well-mixed open compartment. When a drug is administered intravenously, it enters the body and quickly distributes uniformly. The drug then undergoes biotransformation and elimination, ultimately leaving the body. The volume of this compartment is referred to as the apparent volume of distribution into which the drug can uniformly distribute. In this...
3.0K
Mechanistic Models: Overview of Compartment Models
333
Mechanistic models, a category encompassing both physiological and compartmental modeling, differ from empirical models' approaches to incorporating known factors about the systems being modeled. Empirical models describe data with minimal assumptions, while mechanistic models aim to provide a robust description of available data by specifying assumptions and integrating known factors about the system. Compartmental analysis is a key example of a mechanistic model in pharmacokinetics and...
333
Compartment Models: Two-Compartment Model
6.9K
The two-compartment model divides the body into central and peripheral compartments to account for varying blood perfusion rates among organs and tissues, affecting drug distribution. The central compartment includes blood and highly perfused tissues with rapid drug distribution, while the peripheral compartment contains tissues with slower drug distribution. After a single IV bolus dose, the drug concentration is high in plasma and low in tissues. The drug distribution between compartments...
6.9K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
261
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
261
Multi-input and Multi-variable systems
373
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
373


