模型注意力扩展为少数镜头类增量学习
概括
本研究介绍了MTE-FSCIL (Model aTtention Expansion for Few-Shot Class-Incremental Learning) 的注意力扩展模型,以克服监督崩的可能. 新的框架增强了模型的注意力,以更好地转移知识,而不忘记基础类.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算机视觉 计算机视觉
背景情况:
- 少数镜头类增量学习 (FSCIL) 能够使模型在没有忘记先前知识的情况下,在有限的数据上学习新类.
- 现有的FSCIL方法遭受了"监督崩",其中基类特征占主导地位,阻碍了新型类表示和整体模型能力.
- 这种限制限制了AI模型在动态学习环境中的认知能力.
研究的目的:
- 为解决FSCIL监督崩提出一个新的框架,即对少量射击类增量学习 (MTE-FSCIL) 的注意力扩展模型.
- 加强模型关注领域,以提高知识的可转移性,同时保持基层阶级的歧视能力.
- 开发持续知识整合和跨任务学习的策略.
主要方法:
- 一个双阶段的培训策略,包括预培训和超级培训.
- 在预训练期间引入Reserver (RS) 损失以放大特征地图激活,扩大全球感知并减少对类特定特征的依赖.
- 在超级培训期间实施Repeller (RP) 损失,以促进代表性多样性,并通过分散类内样本来改善样本独特性识别.
- 一个转型适应 (TA) 战略,以无地整合下游任务中的新知识.
主要成果:
- 与最先进的方法相比,MTE-FSCIL框架在包括mini-ImageNet,CIFAR100和CUB200在内的基准数据集中表现优越.
- 拟议的RS损失有效地扩大了全球认知,并减轻了过度依赖类特定特征的情况.
- RP损失成功地增强了表示变异,并改善了样本独特性的识别.
- 技术辅助策略有助于有效的跨任务知识转移.
结论:
- MTE-FSCIL有效地克服了监督崩的挑战,在少量射击课堂增量学习中.
- 拟议的框架在增量学习场景中显著提高了模型性能和认知能力.
- MTE-FSCIL框架为开发更强大,更适应的人工智能系统提供了一个有希望的方向.
相关概念视频
01:24Generalization, Discrimination, and Extinction
503
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
503
01:22Multi-input and Multi-variable systems
105
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
105
01:26Per-Unit Sequence Models
73
An ideal Y-Y transformer, grounded through neutral impedances, displays per-unit sequence networks akin to those of a single-phase ideal transformer when subjected to balanced positive- or negative-sequence currents. These currents do not produce neutral currents, and their associated voltage drops.
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
73
01:27Associative Learning
324
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
324
02:07Improving Translational Accuracy
9.7K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
9.7K
01:03Prediction Intervals
2.2K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.2K