一个可扩展的强化学习框架,灵感来自海马体记忆机制,用于高效的上下文和顺序决策.
Hamed Poursiami1, Ayana Moshruba1, Keiland W Cooper2,3
1Department of Electrical and Computer Engineering, George Mason University, Virginia, USA.
Scientific reports
|July 12, 2025
概括
这项研究介绍了海马体增强记忆整合 (HAMI),这是一种基于记忆的强化学习 (RL) 框架. 哈米增强了复杂任务的决策,提高了学习效率和生物灵感的适应性.
科学领域:
- 人工智能的人工智能
- 神经科学是一个神经科学.
- 认知科学 认知科学
背景情况:
- 强化学习 (RL) 中的顺序决策提出了重大挑战,特别是在上下文依赖的任务中.
- 大脑的海马系统提供了一个对记忆整合的模型,这对于有效的学习和适应至关重要.
- 现有的RL方法往往与顺序,丰富上下文环境的复杂性作斗争.
研究的目的:
- 引入海马体增强记忆整合 (HAMI),这是一个生物启发的基于记忆的RL框架.
- 开发层次上下文序列 (HiCoS),这是一个基于神经科学的环境,用于测试基于记忆的决策系统.
- 提高学习效率,适应性和决策准确性,在使用HAMI的上下文依赖的顺序任务中.
主要方法:
- 开发了HAMI,结合了符号索引,层次记忆改进和结构化插曲检索.
- 创建了HiCoS,一个基于情节记忆和上下文驱动的决策原则的结构化的RL环境.
- 评估了HAMI在准确性,样本效率和内存利用方面与基线方法的性能.
主要成果:
- 哈米证明了高决策准确性和提高样本效率.
- 与传统基于内存的RL方法相比,该框架实现了较低的内存利用率.
- 哈米的推理延迟显著降低,这表明硬件加速的潜力.
结论:
- 哈米为复杂的顺序任务提供了一个可扩展和高效的基于内存的RL框架.
- 生物启发的记忆机制和象征性表示的整合是有效的.
- 哈米的架构适合硬件加速,特别是基于非易失性存储器 (NVM) 的内容可定位存储器 (CAM).
相关概念视频
Role of Hippocampus in Memory
556
The hippocampus, a critical brain structure, plays an essential role in memory processing, particularly in the formation and retrieval of memory. This small, seahorse-shaped region is located within the medial temporal lobe, with one hippocampus in each brain hemisphere. Experimental studies involving lesions in the hippocampi of rats have demonstrated significant impairments in tasks such as object recognition and maze navigation, indicating the hippocampus involvement in both recognition and...
556
Storage
135
A schema is a mental framework that helps individuals organize and interpret information. Schemata, formed from previous experiences, influence how we process new information: how we encode it, the inferences we make, and how we retrieve it. For instance, a schema for what a typical classroom looks like might include desks, a teacher's desk, a whiteboard, and students in such an environment. This expectation helps us quickly understand and navigate new classrooms without needing to analyze...
135
Associative Learning
593
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
593
Observational Learning
317
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
317
Real-World Application of Classical Conditioning
742
Classical conditioning not only includes the initial pairing of stimuli but also extends to more complex forms, such as higher-order conditioning. Higher-order conditioning involves creating associations beyond the primary conditioned stimulus, resulting in a chain of conditioned responses.
Higher-order, or second-order, conditioning occurs when a neutral stimulus becomes associated with an already established conditioned stimulus through repeated pairings. For instance, if a dog has been...
Higher-order, or second-order, conditioning occurs when a neutral stimulus becomes associated with an already established conditioned stimulus through repeated pairings. For instance, if a dog has been...
742
Chunking and Rehearsal in Sensory Memory
299
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
299


