事件图指导组合空间-时间推理为视频问题解答
概括
本研究介绍了一个事件图和层次空间时间变压器 (HSTT),用于高级视频问答 (VideoQA). 新方法通过在视频中捕捉多层次的视觉概念,显著改善了构成推理.
科学领域:
- 人工智能的人工智能
- 计算机视觉 计算机视觉
- 机器学习 机器学习
背景情况:
- 视频问题答案 (VideoQA) 需要在多层次视觉概念中进行复杂的构成推理.
- 使用固定持续时间的剪辑特征的现有方法难以以各种细节度捕捉关键概念.
研究的目的:
- 为VideoQA开发一种新的方法,能够有效地代表和推理复杂的视频事件.
- 为了克服固定时间的剪辑功能在捕获多层次视觉信息方面的局限性.
主要方法:
- 使用层次事件图形表示视频,视觉概念 (对象,关系,场景,动作) 的节点和时空关系的边缘.
- 提出一个使用事件图形进行构成推理的层次空间时间变压器 (HSTT).
- 基于语义层次和发生时间的节点编码使用改进的图形搜索算法.
- 引入边缘引导的注意力来整合节点之间的时空环境.
主要成果:
- 拟议的事件图和HSTT方法在具有挑战性的AGQA和STAR数据集上明显优于现有的VideoQA模型.
- 该方法表现出优越的性能,即使与与大型外部数据进行预训练的模型相比.
- 层次表示和注意力机制有效地捕捉多层次的视觉概念,以改善推理.
结论:
- 事件图表表示与HSTT相结合,为视频QA提供了一个强大的框架.
- 这种方法推进了复杂视频理解的构成推理的最新技术.
- 拟议的方法提供了一种更有效的方式来提取和结合视觉概念,以准确地回答视频问题.
更多相关视频
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
15.7K
11:15Automated Visual Cognitive Tasks for Recording Neural Activity Using a Floor Projection Maze
Published on: February 20, 2014
13.1K
相关概念视频
Deductive Reasoning
55.3K
Deductive reasoning, or deduction, is the type of logic used in hypothesis-based science. In deductive reasoning, the pattern of thinking moves in the opposite direction as compared to inductive reasoning, which means that it uses a general principle or law to predict specific results. From those general principles, a scientist can deduce and predict the specific results that would be valid as long as the general principles are valid.
For example, a researcher can deduce specific predictions...
For example, a researcher can deduce specific predictions...
55.3K
Reasoning
77
Reasoning is the action of thinking about something in a logical, sensible way. It is integral to problem-solving, decision-making, and critical thinking. Reasoning can be inductive or deductive. Reasoning involves transforming information into conclusions, which is essential for problem-solving, decision-making, and critical thinking.
Inductive reasoning involves deriving generalizations from specific observations. This type of reasoning helps form beliefs about the world. For example,...
Inductive reasoning involves deriving generalizations from specific observations. This type of reasoning helps form beliefs about the world. For example,...
77
Inductive Reasoning
60.5K
Inductive reasoning is a form of logical thinking that uses related observations to arrive at a general conclusion. It is uncertain and operates in degrees to which the conclusions are credible. As such, inductive arguments can be weak or strong, rather than valid or invalid, and conclusions can be used to formulate testable, falsifiable hypotheses.
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
60.5K
Relative Motion Analysis using Rotating Axes-Problem Solving
404
Consider a crane whose telescopic boom rotates with an angular velocity of 0.04 rad/s and angular acceleration of 0.02 rad/s2. Along with the rotation, the boom also extends linearly with a uniform speed of 5 m/s. The extension of the boom is measured at point D, which is measured with respect to the fixed point C on the other end of the boom. For the given instant, the distance between points C and D is 60 meters.
Here, in order to determine the magnitude of velocity and acceleration for point...
Here, in order to determine the magnitude of velocity and acceleration for point...
404
Frames: Problem Solving II
232
Consider a hydraulic hoist supporting a load of 1 kN. Assuming a simplified schematic representation of this frame structure, the force acting on BD and BF members can be determined.
232
Frames: Problem Solving I
476
Consider a jib crane with an external load suspended from the pulley. The dimensions of the crane members are shown in the figure. A systematic analysis of the frame structure is required to determine the reaction forces at the pin joints, assuming that the pulleys are frictionless.
476
