相关实验视频

Updated: May 24, 2025

The Double-H Maze: A Robust Behavioral Test for Learning and Memory in Rodents

The Double-H Maze: A Robust Behavioral Test for Learning and Memory in Rodents

Published on: July 8, 2015

情节性记忆-双重演员-批评双胞胎延迟深度决定性政策梯度

Man Shu¹, Shuai Lü², Xiaoyu Gong³

¹Key Laboratory of Symbolic Computation and Knowledge Engineering (Jilin University), Ministry of Education, Changchun 130012, China; Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China; College of Computer Science and Technology, Jilin University, Changchun 130012, China.

Neural networks : the official journal of the International Neural Network Society

|March 6, 2025

概括

本研究介绍了情节性记忆 - 双重演员 - 关键 (EMDAC) 框架,以提高持续控制任务的深度强化学习的样本效率. 通过利用情节性记忆来进行行动选择,EMDAC-TD3显著提高了代理的性能和学习速度.

科学领域:

深度强化学习学习 (deep reinforcement learning) 是一种深度强化学习的方法.
机器学习机器学习
机器人和控制系统机器人和控制系统

背景情况:

深度强化学习 (DRL) 算法通常表现出较低的样本效率,限制了它们的实际应用.
插曲记忆在提高离散动作任务的样本效率方面表现有前途,但在高维的连续动作空间中面临挑战.
现有的连续控制方法主要利用情节性记忆信息,而不是直接行动选择指导.

研究的目的:

通过直接使用情节性记忆来进行动作选择,提高持续控制任务中的样本效率.
在固定的培训时间内减少培训步骤或增加奖励.
为此目的,提出一个新的框架,即情节记忆-双重演员-批评 (EMDAC).

主要方法:

提出了情节性记忆-双重演员-批评 (EMDAC) 框架,将情节性记忆集成到连续任务中的行动选择中.
开发了一个使用卡尔曼波器优化器进行基于时间和奖励的加权体验更新的情节性记忆模块.
实现了状态-动作对集群作为情节记忆中的索引,记录事件发生频率和值估计,以实现高效查询.
设计了一个基于状态-动作对的新性基础上的内在奖励机制,以改善代理探索.
将这些模块应用于双延迟深度决定性政策梯度 (TD3) 算法,从而生成EMDAC-TD3算法.

主要成果:

与 MuJoCo 环境中的基线算法相比,EMDAC-TD3 的样本效率显著更高.
该算法在最先进的情节控制和Actor-Critic算法上取得了优越的最终性能.
与TD3相比,EMDAC-TD3的平均性能提高了11.01%,超过了当前最先进的方法.

结论:

在EMDAC框架有效地利用情节性记忆在连续控制任务的行动选择,提高采样效率.
在复杂的控制场景中,EMDAC-TD3提供了一种有希望的方法来提高深度强化学习的性能.
关于情节性记忆管理和内在奖励设计的拟议方法有助于更有能力和更有效的DRL代理.

关键词:

演员关键算法演员关键算法深度强化学习的学习.情节性记忆是一种情节性记忆.样本效率样本效率样本效率样本效率

更多相关视频

Examining Recall Memory in Infancy and Early Childhood Using the Elicited Imitation Paradigm

Examining Recall Memory in Infancy and Early Childhood Using the Elicited Imitation Paradigm

Published on: April 28, 2016

An Operant Intra-/Extra-dimensional Set-shift Task for Mice

An Operant Intra-/Extra-dimensional Set-shift Task for Mice

Published on: January 22, 2016

相关概念视频

Implicit Memories

Implicit Memories

Implicit memories, also known as non-declarative memories, are long-term memories that function outside of conscious awareness. These memories influence behavior and skills without explicit knowledge. This type of memory is evident in tasks like playing tennis, snowboarding, and texting. Implicit memory has three subsystems: procedural memory, conditioning, and priming. This type of memory is essential in various activities, from everyday tasks to specialized skills.
One key aspect of implicit...

Interference and Decay

Interference and Decay

Forgetting is a complex cognitive phenomenon influenced by several factors, among which interference and decay are particularly prominent. These processes explain why individuals often struggle to retrieve specific information from memory, leading to lapses in recall that can be observed in everyday situations.
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...

Reason and Intuition

Reason and Intuition

The human brain processes information for decision-making using one of two routes: an intuitive system and a rational system (Epstein, 1994; popularized by Kahneman, 2011 as System 1 and System 2, respectively). The intuitive system is quick, impulsive, and operates with minimal effort, relying on emotions or habits to provide cues for what to do next, while the rational system is logical, analytical, deliberate, and methodical. Research in neuropsychology suggests that the...

Hindsight Biases

Hindsight Biases

Hindsight bias leads you to believe that the event you just experienced was predictable, even though it really wasn’t. In other words, you knew all along that things would turn out the way they did. Can you relate this to the phrase "Hindsight is 20/20" now?

Traumatic Memory

Traumatic Memory

Emotionally traumatic events often lead to memories that are exceptionally vivid and enduring, sometimes persisting with remarkable clarity throughout an individual's life. A classic example of this phenomenon is a person who survives a car accident. Even years later, they may recall every detail of the event with startling accuracy — the screeching of the tires, the jarring impact, and the acrid smell of burning rubber. Such vividness contrasts sharply with how an individual...

Autobiographical Memory

Autobiographical Memory

Autobiographical memory is a unique type of episodic memory that involves recollecting personal life experiences. It allows individuals to remember significant events from their past, creating a narrative of their lives. One interesting phenomenon related to autobiographical memory is the reminiscence bump. This effect refers to the tendency of adults to recall more events from their second and third decades of life — typically between ages 10 to 30 — than from other periods. This...