双视图对齐学习与等级提示为类不平衡多标签图像分类的等级提示
概括
我们介绍了分层提示双视图对齐学习 (HP-DVAL),以解决多标签图像分类中的类不平衡. 这种方法有效地使用视觉语言模型来提高长尾和少数镜头数据集的性能.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 现实世界数据集经常显示阶级不平衡,导致长尾分布和少数镜头场景.
- 类不平衡多标签图像分类 (CI-MLIC) 任务由于数据不平衡和需要多对象识别而特别具有挑战性.
研究的目的:
- 提出一种新的方法,HP-DVAL,利用视觉语言预训练 (VLP) 模型的多模式知识,以减轻多标签图像分类中的类不平衡.
- 通过分层提示调整策略,提高VLP模型对CI-MLIC任务的适应性.
主要方法:
- 惠普-DVAL利用双视图对齐学习,通过提取图像-文本对齐的互补特征,从VLP模型中转移特征表示功能.
- 使用具有全球和本地提示的等级提示调整策略,以学习特定任务和与上下文相关的先前知识.
- 在提示调整过程中包含一个语义一致性损失,以保持VLP模型中的一般知识完整性.
主要成果:
- 拟议的HP-DVAL方法在两个CI-MLIC基准指标上表现出卓越的表现:MS-COCO和VOC2007.
- 与最先进的方法相比,在平均平均精度 (mAP) 中实现了显著的改进:在长尾多标签图像分类任务中分别为10.0%和5.2%,在多标签少数镜头图像分类任务中分别为6.8%和2.9%.
结论:
- 通过利用VLP模型和一种新的提示调整策略,HP-DVAL有效地解决了多标签图像分类中的类不平衡问题.
- 该方法显示了在具有挑战性的现实世界场景中提高性能的巨大潜力,这些场景的特点是数据不平衡和少量学习.
相关概念视频
Force Classification
2.3K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
2.3K
Multi-input and Multi-variable systems
394
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
394
Classification of Systems-II
460
Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
460
Associative Learning
1.3K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.3K
Observational Learning
841
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
841
Aggregates Classification
970
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
970

