因果引导的共同注意网络用于视觉问题答案
Jiali Miao1, Kui Yu1, Baofu Fang2
1Key Laboratory of Knowledge Engineering with Big Data (the Ministry of Education of China), Hefei University of Technology, Hefei, 230601, China; School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, 230601, China.
概括
因果引导协同注意网络 (CGCN) 通过区分有影响力的和支持的对象特征来改善视觉问题答案 (VQA). 这种方法增强了多式联络学习,以更好地基于图像回答问题.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 自然语言处理自然语言处理.
背景情况:
- 视觉问题答案 (VQA) 对多式联络人工智能至关重要,它利用视觉和语言理解.
- 当前的VQA模型与杂乱的图像扎,因为自我注意力机制无法将注意力集中在相关对象上.
- 专注于相关对象可能是有害的,而背景特征有时可以帮助性能.
研究的目的:
- 解决复杂视觉场景的VQA模型中自我注意的局限性.
- 开发一个新的网络,有效地利用相关和支持的视觉特征.
- 通过改进多式模式表示学习来提高VQA系统的准确性.
主要方法:
- 提出了因果导向协同注意网络 (CGCN),这是一个层次化的颗粒状视觉特征注意网络.
- 引入了一个因果图来建模对象特征关系,将它们分类为有影响的对象特征 (IOF) 和支持的对象特征 (SOF).
- 开发了一种因果关系启发的共同注意力机制,以因果图为指导,用于交叉模式学习和双输出融合方法.
主要成果:
- 在视觉问题答案任务中,CGCN显著提高了性能.
- 拟议的方法显示了对相关视觉信息的关注能力的提高,同时结合了支持功能.
- 实验结果验证了VQA的因果关系引导方法的有效性.
结论:
- CGCN提供了一种有希望的方法来克服VQA中的挑战,特别是在许多重叠对象的场景中.
- 区分和利用IOF和SOF有效地提高了多式联运特征表示和融合.
- 因果导向的共同注意力机制代表了VQA模型开发的重大进步.
相关概念视频
01:25The Anchoring-and-Adjustment Heuristic
7.7K
In order to make good decisions, we use our knowledge and our reasoning. Often, this knowledge and reasoning is sound and solid. However, sometimes, we are swayed by biases or by others manipulating a situation. For example, let’s say you and three friends wanted to rent a house and had a combined target budget of $1,600. The realtor shows you only very run-down houses for $1,600 and then shows you a very nice house for $2,000. Might you ask each person to pay more in rent to get the...
7.7K
01:27Associative Learning
1.2K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.2K
01:12Observational Learning
824
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
824
01:12Visual Agnosia
957
Visual agnosia is a condition characterized by the inability to recognize visually presented objects despite having normal vision. For instance, a person with visual agnosia can describe the shape and color of an object but cannot identify or name it. This impairment does not affect their visual field, acuity, color vision, brightness discrimination, language, or memory. An example of this condition in a social setting is someone at a dinner party asking for "that silver thing with a round...
957
01:26Visual System
1.7K
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
1.7K