通过视频图形变压器回答对比视频问题
概括
我们介绍了CoVGT,这是一个新的视频图形转换器模型,用于视频问答 (VideoQA). CoVGT在时空推理方面表现出色,并实现了卓越的性能,甚至超过了在庞大的数据集上预训练的模型.
科学领域:
- 人工智能的人工智能
- 计算机视觉 计算机视觉
- 自然语言处理自然语言处理.
背景情况:
- 视频问答 (VideoQA) 是一个具有挑战性的任务,需要理解复杂的时空关系.
- 现有的视频QA模型经常在细微的推理和高效的跨模式理解方面扎.
研究的目的:
- 为增强视频QA提出一个新的视频图形变压器模型 (CoVGT).
- 为了改善视频QA任务的时空推理和视频文本对比学习.
主要方法:
- 开发了一个动态图形转换器模块,通过捕获对象,关系和动态来编码视频.
- 采用了独立的视频和文本转换器,并配备了用于对比学习的跨模式交互模块.
- 使用联合完全和自我监督的对比目标进行优化.
主要成果:
- 与先前的技术相比,CoVGT在视频推理任务上表现出更好的表现.
- 与在显著更大的数据集上预训练的模型相比,取得了更好的结果.
- 通过数量级减少跨模式预训练数据来展示有效性.
结论:
- CoVGT 为视频QA.QA提供了一个高效和优质的解决方案.
- 该模型显示出更多数据效率高的预训练策略的潜力.
- 强调了视频QA中显式时空编码和对比学习的优势.
更多相关视频
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
15.8K
08:25Combining Eye-tracking Data with an Analysis of Video Content from Free-viewing a Video of a Walk in an Urban Park Environment
Published on: May 7, 2019
9.0K
相关概念视频
Types Of Transformers
1.0K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.0K
Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K
The Ideal Transformer
434
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
434
Transformers with Off-Nominal Turns Ratios
182
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
182
Transformers in Distribution System
127
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
127
Transformation
33
Microbial communities are dynamic environments where cell lysis releases free DNA into the surroundings. Other cells can take up this extracellular DNA through a process known as transformation.When a cell incorporates this foreign DNA into its genome, resulting in genetic modification, the process is known as transformation. Cells capable of this process are termed competent. Competence can be natural, as observed in certain bacteria and archaea, or artificially induced in the...
33
