在寻找分散的记忆:生成扩散模型是关联式记忆网络
1Donders Institute for Brain, Cognition, and Behaviour, Radboud University, 6525 XZ Nijmegen, The Netherlands.
Entropy (Basel, Switzerland)
|May 24, 2024
概括
生成性扩散模型相当于用于关联性记忆的现代霍普菲尔德网络. 这一发现将创造性生成和记忆回忆统一在一个单一的框架内,以了解长期记忆的形成.
科学领域:
- 神经科学是一个神经科学.
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 了解长期记忆机制是神经科学和人工智能的关键挑战.
- 人工关联性记忆网络模拟生物记忆的各个方面.
- 生成式扩散模型在各种机器学习任务中表现出色.
研究的目的:
- 将生成扩散模型解释为基于能源的模型.
- 为了建立扩散模型和现代霍普菲尔德网络之间的等价性.
- 为长期记忆,创造性生成和回忆开发一个统一的框架.
主要方法:
- 将生成扩散模型解释为基于能源的模型.
- 在离散模式上训练扩散模型.
- 分析训练的扩散模型的能量函数.
- 将能量功能与现代霍普菲尔德网络的能量功能进行比较.
主要成果:
- 生成性扩散模型可以被视为基于能源的模型.
- 当在离散模式上训练时,它们的能量功能与现代霍普菲尔德网络相同.
- 扩散模型的监督训练与Hopfield网络中的突触学习相对应.
结论:
- 生成性扩散模型和现代霍普菲尔德网络之间存在着强烈的理论联系.
- 这种联系为突触学习和关联动态提供了新的视角.
- 建议为长期记忆提供一个统一的框架,包括生成和回忆.
相关概念视频
Storage
84
A schema is a mental framework that helps individuals organize and interpret information. Schemata, formed from previous experiences, influence how we process new information: how we encode it, the inferences we make, and how we retrieve it. For instance, a schema for what a typical classroom looks like might include desks, a teacher's desk, a whiteboard, and students in such an environment. This expectation helps us quickly understand and navigate new classrooms without needing to analyze...
84
Higher Mental Functions of Brain: Learning and Memory
744
Memory is one of the most vital higher mental functions of the brain. Memory is closely related to learning because it enables us to retain information and experiences from our past to use them in our present life. It also helps us to remember facts, events, and skills, such as riding a bike or swimming. There are two types of memory — declarative memory, which involves memorizing facts or events, and procedural memory, which enables us to remember how to do something like writing or...
744
Interference and Decay
135
Forgetting is a complex cognitive phenomenon influenced by several factors, among which interference and decay are particularly prominent. These processes explain why individuals often struggle to retrieve specific information from memory, leading to lapses in recall that can be observed in everyday situations.
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...
135
Neural Circuits
1.2K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.2K
Dissociative Disorders
65
Dissociative disorders represent complex psychological conditions characterized by disruptions in consciousness, memory, identity, or perception. These disruptions cause individuals to experience a disconnection from their thoughts, emotions, and memories. The phenomenon is not merely an occasional lapse in attention but a profound alteration in mental functioning that can severely impact daily life.
Dissociative Fugue
A hallmark feature of dissociative disorders is the dissociative fugue...
Dissociative Fugue
A hallmark feature of dissociative disorders is the dissociative fugue...
65
Implicit Memories
127
Implicit memories, also known as non-declarative memories, are long-term memories that function outside of conscious awareness. These memories influence behavior and skills without explicit knowledge. This type of memory is evident in tasks like playing tennis, snowboarding, and texting. Implicit memory has three subsystems: procedural memory, conditioning, and priming. This type of memory is essential in various activities, from everyday tasks to specialized skills.
One key aspect of implicit...
One key aspect of implicit...
127


