色的记忆作为残余循环网络中的感应偏差
Igor Dubinin1, Felix Effenberger2
1Ernst Strüngmann Institute, Deutschordenstraße 46, Frankfurt am Main, 60528, Germany; Frankfurt Institute for Advanced Studies, Ruth-Moufang-Straße 1, Frankfurt am Main, 60438, Germany.
概括
弱合的剩余循环网络 (WCRNNs) 利用剩余连接来改善循环神经网络 (RNN) 动态和内存. 这项研究探讨了这些连接如何提高网络表达力和基准任务的性能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算神经科学是一种神经科学.
背景情况:
- 已知剩余连接可以通过减轻梯度问题来改善前和循环神经网络 (RNN) 的训练.
- 剩余连接对RNN动态和色内存属性的特定影响仍然在很大程度上未被探索.
研究的目的:
- 引入和分析弱合的剩余循环网络 (WCRNNs),以了解剩余连接对RNN动态和内存的影响.
- 调查WCRNN如何影响跨基准任务的网络性能,动态和内存属性.
主要方法:
- 开发了包含残余连接的WCRNN,旨在产生明确的Lyapunov指数用于记忆分析.
- 在一系列基准任务中评估WCRNN性能,网络动态和内存属性.
- 将发现扩展到非线性余量,并为埃尔曼RNN提出了一个弱合的余量初始化.
主要成果:
- 证明了WCRNN中不同形式的残余连接提供了有效的诱导偏差,增强了网络表达性.
- 展示了剩余连接,促进了接近混乱边缘的动态,使网络能够利用数据的光谱属性,并诱导异质记忆属性.
- 证实了这些发现对非线性残留的适用性,并为埃尔曼RNN引入了一个新的初始化方案.
结论:
- 在RNN中的剩余连接,特别是在WCRNN框架内,显著提高了网络的表达性和性能.
- WCRNN为研究和优化RNN内存属性和动态提供了一种有价值的方法.
- 拟议的弱合剩余初始化方案为改善Elman RNNs提供了一种实际方法.
相关概念视频
Interference and Decay
140
Forgetting is a complex cognitive phenomenon influenced by several factors, among which interference and decay are particularly prominent. These processes explain why individuals often struggle to retrieve specific information from memory, leading to lapses in recall that can be observed in everyday situations.
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...
140
Forgetting
77
Forgetting is an intrinsic aspect of human memory, characterized by the gradual loss or inaccessibility of information over time. Hermann Ebbinghaus, a pioneering psychologist, extensively studied this phenomenon and formulated the forgetting curve. This curve illustrates that memory loss occurs rapidly immediately after learning and then decelerates over time. Several mechanisms contribute to forgetting, including encoding failure, storage decay, retrieval failure, and interference.
Encoding...
Encoding...
77
Hindsight Biases
3.4K
Hindsight bias leads you to believe that the event you just experienced was predictable, even though it really wasn’t. In other words, you knew all along that things would turn out the way they did. Can you relate this to the phrase "Hindsight is 20/20" now?
3.4K
Higher Mental Functions of Brain: Learning and Memory
799
Memory is one of the most vital higher mental functions of the brain. Memory is closely related to learning because it enables us to retain information and experiences from our past to use them in our present life. It also helps us to remember facts, events, and skills, such as riding a bike or swimming. There are two types of memory — declarative memory, which involves memorizing facts or events, and procedural memory, which enables us to remember how to do something like writing or...
799
Implicit Memories
130
Implicit memories, also known as non-declarative memories, are long-term memories that function outside of conscious awareness. These memories influence behavior and skills without explicit knowledge. This type of memory is evident in tasks like playing tennis, snowboarding, and texting. Implicit memory has three subsystems: procedural memory, conditioning, and priming. This type of memory is essential in various activities, from everyday tasks to specialized skills.
One key aspect of implicit...
One key aspect of implicit...
130
Associative Learning
362
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
362


