相关实验视频
Updated: May 28, 2025

Cross-Modal Multivariate Pattern Analysis
Published on: November 9, 2011
交叉模式的配方检索与细粒度的模式交互
Fan Zhao1, Yuqing Lu2, Zhuo Yao2
1Faculty of Printing, Packaging Engineering and Digital Media Technology &State Key Laboratory of Eco-hydraulics in Northwest Arid Region, Xi'an University of Technology, Xi'an, 710048, China. vcu@xaut.edu.cn.
本研究介绍了用于跨模态配方检索的细粒度模式交互 (FMI). FMI显著改善了与食谱相匹配的食物图像,反之亦然,优于现有的方法.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 交叉模式的食谱检索,将食物图像与食谱相匹配,反之亦然,是一个不断增长的研究领域.
- 现有的方法往往缺乏模式之间的细粒度交互,限制了多模式表示增强.
- 先进的技术已经改善了一般的基准性能,但需要更深层次的模式交互.
研究的目的:
- 通过利用微粒度模式的交互来增强多模式表示,以实现跨模式的配方检索.
- 引入新的模块来丰富配方组件和视觉表示.
- 在标准数据集上验证拟议方法的有效性.
主要方法:
- 开发了跨组件的多尺度配方丰富 (CCMRE) 模块,使用层次配方变压器和完全卷积运算.
- 在图像编码器中引入了文本上下文化视觉增强 (TCVE) 模块,以进行更深入的视觉表示重新学习.
- 利用图像局部特征和中间配方表示之间的相似性进行增强.
主要成果:
- 拟议的细粒度模式交互 (FMI) 方法在Recipe1M数据集上表现出卓越的性能.
- 与基线模型相比,FMI取得了显著的改进,在1k测试组中达到+17.4 R@1,在10k测试组中达到+20.5 R@1.
- 废弃性研究证实了拟议的CCMRE和TCVE模块的有效性.
结论:
- FMI 方法通过细粒度交互有效地增强了多模式表示,推进了跨模式的配方检索.
- 新的CCMRE和TCVE模块通过丰富单个模式及其相互作用,有助于提高性能.
- 这项研究为跨模式的配方检索任务设定了新的最先进的状态.
更多相关视频
06:43The Crossmodal Congruency Task as a Means to Obtain an Objective Behavioral Measure in the Rubber Hand Illusion Paradigm
Published on: July 26, 2013
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
相关概念视频
Sensory Modalities
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
Optimal Foraging
Crossover Experiments
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
Cross-reactivity
Taste Buds and Receptors
Woodward–Hoffmann Selection Rules and Microscopic Reversibility