一个快速的规范化方法,以增强几次拍摄的类增量学习与双阶段分类器
Meilan Hao1, Yizhan Gu2, Kejian Dong2
1School of Information and Electrical Engineering, Hebei University of Engineering, Handan, 056038, China; Institute of Semiconductors, Chinese Academy of Sciences, Beijing, 100083, China.
概括
本研究介绍了为少量射击类增量学习 (FSCIL) 的快速规范化 (PrRe). 这种新的方法增强了预先训练的视觉转换器 (ViTs),以提高模型的性能,而不忘记之前的任务.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 低射门班级增量学习 (FSCIL) 解决了有限数据的培训模型和持续的学习,而不是灾难性的遗忘.
- 预训练的模型提供了强大的特征表示和可转移性,这对于少量拍摄和增量学习至关重要.
- 快速学习可以提高预训练模型在下游任务中的性能,特别是在大规模的视觉和语言模型中.
研究的目的:
- 提出一种新的快速规范化 (PrRe) 方法,用于少量射击类增量学习 (FSCIL).
- 通过融合任务和全球提示来增强预先训练有素的视觉转换者 (ViT).
- 提高模型效率,防止在增量学习场景中知识被遗忘.
主要方法:
- 开发了一种快速规范化 (PrRe) 方法,将任务和全球提示嵌入预训练的视觉转换器 (ViT) 中.
- 引入了一种两阶段分类器 (TSC),用于基础会话使用K-Nearest Neighbors,以及用于增量会话的原型分类器.
- 在分类阶段内集成了一个全局自我注意模块.
主要成果:
- 通过对多个基准数据集的实验,证明了拟议的PrRe方法的有效性.
- 展示了 PrRe 方法在 Few-Shot 类增量学习任务中的优势.
- 验证了该方法有效更新模型的能力,而不会忘记以前学习的信息.
结论:
- 提议的快速规范化 (PrRe) 方法有效地增强了FSCIL预先训练的视觉转换器.
- 与全球自我注意力集成的双阶段分类器 (TSC) 在增量学习会话中被证明是有效的.
- 该方法提供了一个有前途的解决方案,以有限的数据提供高效和稳定的增量学习.
相关概念视频
01:24Generalization, Discrimination, and Extinction
347
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
347
02:07Improving Translational Accuracy
8.5K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
8.5K
01:22Multi-input and Multi-variable systems
89
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
89
01:31Classification of Systems-II
125
Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
125
01:22Force Classification
1.0K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.0K
01:03Prediction Intervals
2.2K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.2K