调整组合特征重播,以实现高效的学习流
IEEE transactions on neural networks and learning systems
|December 25, 2023
概括
人工神经网络在持续学习方面扎,但一种新的算法CRUMB (使用内存块的组成重复播放) 通过重建和重复播放特征地图来缓解灾难性遗忘,从而改善流学习中的知识保留.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算机视觉 计算机视觉
背景情况:
- 人工神经网络 (ANN) 在从顺序数据中提取可概括的知识方面存在局限性,与人类大脑不同.
- 在混合数据上训练的标准ANN在学习新信息时遭受灾难性的遗忘 (在线流学习).
研究的目的:
- 引入一种新的持续学习算法,使用内存块 (CRUMB) 进行组合重复,旨在减轻ANN中的灾难性遗忘.
- 增强ANN保留过去经验的知识的能力,同时在连续流中学习新的刺激.
主要方法:
- CRUMB通过结合在卷积神经网络 (CNN) 中可训练和可重复使用的"内存块"中存储的通用部分来重建特征地图.
- 算法存储记忆块的索引,以便在以后的任务中重复刺激,将网络偏向形状信息和稳定训练.
- 在七个数据集上,CRUMB与13种竞争方法进行了评估,其中包括两种新适应的在线流学习基准.
主要成果:
- 在减轻灾难性遗忘方面,CRUMB显著优于现有方法,在显著减少内存的情况下实现更高的性能 (3.6%的可比图像重播算法).
- 与最先进的方法相比,该算法以最小的内存开销 (3.7%-4.1%) 和可比的运行时间 (15%-43%) 证明了有效性.
- CRUMB为培训示例提供了一个稳定的,共享的特征级别基础,提高了概括性,减少了新的,未见过的数据的影响.
结论:
- CRUMB为ANN的持续学习提供了一种高效和高效的记忆解决方案,解决了灾难性遗忘的关键挑战.
- 功能地图的组成重建为开发更强大,更适应的人工智能系统提供了一个有希望的方向.
- 拟议的方法显著推进了在线流学习领域,并为人工智能研究和开发提供了宝贵的工具.
相关概念视频
Elaborative Rehearsals
88
Elaborative rehearsal is a crucial cognitive strategy that strengthens information encoding in long-term memory by making meaningful connections between new data and pre-existing knowledge. This approach contrasts with maintenance rehearsal, which involves simple repetition without delving into the significance of the information. While maintenance rehearsal might temporarily keep information active in short-term memory, it is less effective for long-term retention.
The effectiveness of...
The effectiveness of...
88
Chunking and Rehearsal in Sensory Memory
215
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
215
Downsampling
159
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
159
Stream Function
1.5K
In two-dimensional incompressible fluid flow, the continuity equation is essential for ensuring mass conservation, meaning that any change in fluid entering or exiting a region is balanced by a corresponding change elsewhere. For incompressible flow, where density remains constant, this requirement simplifies to the condition that the divergence of the velocity field must be zero. Mathematically, this is expressed as,
1.5K
Sampling Plans
187
Sampling is a crucial step in analytical chemistry, allowing researchers to collect representative data from a large population. Common sampling methods include random, judgmental, systematic, stratified, and cluster sampling.
Random sampling is a method where each member of the population has an equal chance of being selected for the sample. It involves selecting individuals randomly, often using random number generators or lottery-type methods. For example, when analyzing the properties of a...
Random sampling is a method where each member of the population has an equal chance of being selected for the sample. It involves selecting individuals randomly, often using random number generators or lottery-type methods. For example, when analyzing the properties of a...
187
Sampling Continuous Time Signal
251
In signal processing, a continuous-time signal can be sampled using an impulse-train sampling technique, followed by the zero-order hold method. Impulse-train sampling involves the use of a periodic impulse train, which consists of a series of delta functions spaced at regular intervals determined by the sampling period. When a continuous-time signal is multiplied by this impulse train, it generates impulses with amplitudes corresponding to the signal's values at the sampling points.
In the...
In the...
251


