自主训练增强了复合图像检索的多因素匹配网络
IEEE transactions on pattern analysis and machine intelligence
|December 25, 2023
概括
本研究引入了一种用于复合图像检索 (CIR) 的新方法,该方法考虑了多个匹配因素和未标记的数据. 拟议的LIMN和LIMN+模型通过解决现有方法的关键局限性,显著提高了检索准确性.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 复合图像检索 (CIR) 涉及从多式联络查询 (参考图像+文本) 中找到目标图像.
- 现有的CIR方法往往忽略了多个匹配因素,并未充分利用未标记的数据,从而限制了性能.
- 挑战包括在没有监督的情况下建模潜在匹配因子,并利用未标记的对进行更好的概括.
研究的目的:
- 通过开发一个能够考虑多个匹配因子和未标记数据的模型来解决CIR的局限性.
- 为提高CIR性能提出一种新的网络架构和培训范式.
- 提高CIR模型的一般化能力.
主要方法:
- 提出了一个基于CLIP转换器的多因素匹配网络 (LIMN),其中包括用于潜在因子挖掘,匹配令牌学习和查询目标匹配的模块.
- 引入了一种代的双重自我训练范式 (LIMN+),以弱监督的方式利用未标记的参考目标图像对.
- 在FashionIQ,Shoes,CIRR和Fashion200K数据集上评估了业绩.
主要成果:
- LIMN有效地以潜在的方式模拟多个匹配因素.
- 通过自我训练,LIMN+通过利用未标记的数据显著提高了CIR性能.
- 无论是LIMN还是LIMN+,在多个数据集中都显示出相对于最先进的基线的显著改进.
结论:
- 拟议的LIMN和LIMN+模型在复合图像检索方面取得了重大进展.
- 有效地建模多个因素并利用未标记的数据对于改进CIR至关重要.
- 开发的方法显示了对需要细微图像检索的现实世界的应用有很大的潜力.
相关概念视频
Multi-input and Multi-variable systems
106
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
106
Aggregates Classification
327
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
327
Force Classification
1.2K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.2K
Wilcoxon Signed-Ranks Test for Matched Pairs
134
The Wilcoxon signed-rank test for matched pairs evaluates the null hypothesis by combining the ranks of differences with their signs. It essentially tests whether the median of the differences in a population of matched pairs is zero. Since the test incorporates more information than the sign test, it generally yields more trustable conclusions. This test also does not require the data to follow a normal distribution, but two conditions must be met for it to be applicable: (1) the data must...
134
Improving Translational Accuracy
10.8K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
10.8K
Sign Test for Matched Pairs
134
The sign test for matched pairs offers a robust method for comparing two paired samples, often for the effects of an intervention in one of them. This method is very useful in situations where the underlying distribution of the data is unknown. The test compares two related samples—often pre- and post-treatment measurements on the same subjects—to determine if there are significant differences in their median values.
To conduct the sign test, we first calculate the differences in...
To conduct the sign test, we first calculate the differences in...
134


