相互上下文关系引导的动态图形网络,用于跨模式的图像-文本检索
G Sucharitha1, B J D Kalyani2, Akella S Narasimha Raju3
1Department of Computer Science and Engineering, Anurag University, Hyderabad, Telangana, India. sucharithasu@gmail.com.
Scientific reports
|October 1, 2025
概括
本研究介绍了一种用于跨模式检索的新型动态图形网络,通过建模相互上下文关系来增强图像-文本匹配. 这种方法显著提高了在各种模式中检索语义相关内容的精度和回忆.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 跨模式检索对于多媒体搜索和推至关重要,因为多模式数据的增加.
- 挑战包括图像和文本表示之间的异质性和语义差距.
- 现有的模型经常与静态特征对齐和上下文关系的不充分建模作斗争.
研究的目的:
- 提出一种新的相互上下文关系引导的动态图形网络,用于统一和可解释的多式联络表示.
- 通过动态调整视觉和文本特征来增强图像-文本匹配.
- 克服现有的跨模式检索方法的局限性.
主要方法:
- 视觉变压器 (ViT),BERT和图形卷积神经网络 (GCNN) 的集成.
- 构建一个动态交叉模式特征图 (DCMFG),其节点代表图像和文本特征.
- 基于相互上下文关系 (KNN) 的动态边缘更新和以注意力为导向的适应性调整机制.
主要成果:
- 在基准数据集 (MirFlickr-25K, NUS-WIDE) 的精度和回忆方面显著提高了性能.
- 在交叉模式检索中,与最先进的方法相比,经过证明的有效性.
- 通过揭示图像区域和文本特征之间的相互作用来提高可解释性.
结论:
- 拟议的动态图形网络有效地解决了跨模式检索的挑战.
- 该方法为多式模式表示学习提供了一个强大的和可解释的方法.
- 对准确和语义相关的跨模式检索的验证有效性.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.3K
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
1.0K
相关概念视频
01:06Collisions in Multiple Dimensions: Problem Solving
5.3K
In multiple dimensions, the conservation of momentum applies in each direction independently. Hence, to solve collisions in multiple dimensions, we should write down the momentum conservation in each direction separately. To help understand collisions in multiple dimensions, consider an example.
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
5.3K
01:45ER Retrieval Pathway
4.7K
In the secretory pathway, vesicles transport proteins from one cellular compartment to another in forward transport to deliver the protein to its correct location. Occasionally, misfolded proteins and incorrect proteins escape their original compartments, and a retrieval pathway is used to return the escaped proteins to their original compartment.
The ER uses many checkpoints to prevent the entry of incorrectly folded or a resident protein as cargo onto a transport vesicle. These mechanisms...
The ER uses many checkpoints to prevent the entry of incorrectly folded or a resident protein as cargo onto a transport vesicle. These mechanisms...
4.7K