基于地图的体验重复:在强化学习中,对灾难性遗忘的记忆效率高的解决方案
Muhammad Burhan Hafez1, Tilman Immisch1, Tom Weber1
1Knowledge Technology Research Group, Department of Informatics, University of Hamburg, Hamburg, Germany.
Frontiers in neurorobotics
|July 13, 2023
概括
这项研究引入了用于深度强化学习 (RL) 的认知灵感重复记忆. 这种新的方法通过组织过渡到一个网络来减少内存大小,减轻灾难性遗忘.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 认知科学 认知科学
背景情况:
- 深度强化学习 (RL) 代理人容易发生灾难性遗忘,新培训数据会覆盖以前学到的知识.
- 传统的重播记忆存储状态转换而不考虑冗余性,导致效率低下的内存使用.
研究的目的:
- 开发一种新的认知灵感重复记忆系统,用于深度强化学习.
- 解决灾难性遗忘问题,提高RL代理的记忆效率.
主要方法:
- 实现了基于Grow-When-Required (GWR) 自组织网络的重播记忆方法.
- 组织存储状态转换成状态节点和过渡边缘网络,合并类似的样本.
- 利用基于地图的心理模型来结构体验重复.
主要成果:
- 拟议的方法通过创建一个简洁的,类似环境模型的网络,显著减少了内存大小.
- 合并类似的样本增加了存储过渡之间的相关性和对距离.
- 实现显著的记忆减少,只有轻微的性能下降.
结论:
- 基于地图的体验重复提供了一个有效的策略,以减少深度强化学习的记忆.
- 受认知启发的GWR网络为缓解灾难性遗忘提供了一个有希望的方向.
- 这种方法在RL代理中平衡了记忆效率与学习性能.
相关概念视频
Interference and Decay
172
Forgetting is a complex cognitive phenomenon influenced by several factors, among which interference and decay are particularly prominent. These processes explain why individuals often struggle to retrieve specific information from memory, leading to lapses in recall that can be observed in everyday situations.
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...
172
Forgetting
100
Forgetting is an intrinsic aspect of human memory, characterized by the gradual loss or inaccessibility of information over time. Hermann Ebbinghaus, a pioneering psychologist, extensively studied this phenomenon and formulated the forgetting curve. This curve illustrates that memory loss occurs rapidly immediately after learning and then decelerates over time. Several mechanisms contribute to forgetting, including encoding failure, storage decay, retrieval failure, and interference.
Encoding...
Encoding...
100
Repressed Memory
121
Repressed memories are a psychological phenomenon where memories of traumatic events are unconsciously blocked from a person's awareness. This process occurs as a defense mechanism, protecting the mind from the emotional impact of distressing or painful experiences. For example, a person who has experienced childhood trauma may grow up with no conscious recollection of the event. In such cases, the memories are thought to be buried deep within the subconscious, inaccessible to the conscious...
121
Chunking and Rehearsal in Sensory Memory
247
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
247
Traumatic Memory
132
Emotionally traumatic events often lead to memories that are exceptionally vivid and enduring, sometimes persisting with remarkable clarity throughout an individual's life. A classic example of this phenomenon is a person who survives a car accident. Even years later, they may recall every detail of the event with startling accuracy — the screeching of the tires, the jarring impact, and the acrid smell of burning rubber. Such vividness contrasts sharply with how an individual...
132
Avoidance Learning and Learned Helplessness
1.8K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
1.8K


