变形金刚将视觉和语言联系起来,以估计和理解场景含义
Taylor R Hayes1, John M Henderson1,2
1Center for Mind and Brain, University of California, Davis.
Research square
|July 3, 2023
概括
最先进的多式联络变压器现在可以估计场景的含义,并预测人类的注意力. 这种方法为我们提供了关于语义知识如何引导我们理解复杂的视觉环境的新见解.
科学领域:
- 认知科学 认知科学
- 计算机视觉 计算机视觉
- 人工智能的人工智能
背景情况:
- 人类对场景的理解依赖于语义知识来组织感知和引导注意力.
- 存储的语义表示在场景指导中的确切作用尚不清楚.
- 现有的方法很难模拟语义和视觉处理之间的相互作用.
研究的目的:
- 通过使用先进的人工智能,研究语义表示在场景理解中的作用.
- 探索AI如何在复杂场景中模拟和预测人类视觉注意力.
- 开发用于分析场景意义及其对感知的影响的计算方法.
主要方法:
- 使用一台经过广泛图像文本数据集训练的最先进的多式联络变压器.
- 应用变压器在各种室内和室外环境中自动估计局部场景的含义.
- 评估了模型在场景内预测人类眼睛运动 (凝视预测) 的能力.
- 开发了检测语义变化的方法,并为场景区域突出提供可解释的解释.
主要成果:
- 多式联网变压器成功地估计了各种环境中的本地场景意义.
- 该模型准确地预测了在测试场景中的人类视觉注意力模式.
- 该方法在检测语义转移和解释视觉突出性方面表现出能力.
- 研究结果表明,人工智能可以作为一个框架,在场景理解中弥合视觉和语言.
结论:
- 多模转换器作为一个强大的表示框架,用于研究场景语义.
- 这些模型提升了我们对语义知识如何影响视觉场景感知和注意力的理解.
- 桥梁视觉和语言处理为认知科学研究提供了新的途径.
相关概念视频
01:27Transformers in Distribution System
127
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
127
01:26Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K
01:24Vision
53.6K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.6K
01:25Transformers with Off-Nominal Turns Ratios
182
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
182
01:26Visual System
623
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
623
01:16Types Of Transformers
1.0K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.0K