循环一致的弱监督视觉接地与个体和上下文表示
概括
这项研究引入了循环一致性,用于弱监督的视觉接地,减少图像-文本对齐的模糊性. 新的并行框架通过结合个人和上下文表示来提高准确性来增强跨模式学习.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 视觉接地将图像区域与文本连接起来,这对于跨模式学习至关重要.
- 弱监督的视觉接地缺乏细粒度的对应,导致匹配的模糊性.
研究的目的:
- 为了解决在弱监督的视觉接地中匹配的模糊性.
- 为了提高图像区域和文本查询之间的对齐精度.
主要方法:
- 引入了一个周期一致性约束区域-短语对加强正确的匹配和削弱不正确的.
- 提出了一个平行接地框架,提取个人和上下文表示.
- 计算了区域和短语的无上下文和上下文意识的相似性.
- 使用图像-文本对比损失和循环一致性损失训练框架.
主要成果:
- 在五个基准数据集中实现了可见的视觉接地性能改进.
- 证明了循环一致性在缓解匹配模糊性的有效性.
- 展示了个人和上下文表示的互补性质,以实现跨模式对齐.
结论:
- 拟议的方法有效地增强了弱监督的视觉接地.
- 循环一致性和并行框架为跨模式对齐提供了一个强大的解决方案.
- 该方法在微弱监督下,在精确地将文本与图像接地方面迈出了重要的一步.
更多相关视频
相关概念视频
Gestalt Principles of Perception
340
Gestalt principles provide a framework for understanding how humans perceive objects as unified wholes within their context. These principles are essential in explaining the cognitive processes that make sense of complex visual stimuli by organizing them into coherent groups. One fundamental principle is proximity, which posits that objects located close to each other are perceived as a collective group. For instance, when dots are positioned near one another, the visual system interprets them...
340
The Representativeness Heuristic
15.8K
The representative heuristic describes a biased way of thinking, in which you unintentionally stereotype someone or something. For example, you may assume that your professors spend their free time reading books and engaging in intellectual conversation, because the idea of them spending their time playing volleyball or visiting an amusement park does not fit in with your stereotypes of professors.
15.8K
Perceptual Constancy
437
Perceptual constancy is the ability to recognize that objects remain consistent and unchanged even when their appearance varies due to changes in sensory input. There are four main types of perceptual constancy: size constancy, shape constancy, color constancy, and brightness constancy.
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
437
Natural and Artificial Concepts
189
In psychology, concepts can be divided into two categories: natural and artificial. Natural concepts are formed through direct or indirect experiences. For example, consider the concept of snow. If you live in a place with regular snowfall, such as Essex Junction, Vermont, you know snow through direct experiences. You’ve seen it fall, touched it, shoveled it, and played in it. You recognize its texture, appearance, and even its smell. In contrast, if you live on an island like Saint...
189
Depth Perception and Spatial Vision
709
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
709
Observational Learning
207
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
207


