面向事件的状态对准网络,用于弱监督的时间语言接地
Hongzhou Wu1, Xiang Zhang1, Tao Tang1
1School of Computer, National University of Defense Technology, Changsha 410073, China.
Entropy (Basel, Switzerland)
|September 27, 2024
概括
本研究介绍了以事件为导向的状态调整网络 (ESAN),用于弱监督的时间语言接地 (TLG). 通过在文本和视频中对准语义状态,ESAN 改进了视频中的事件本地化,优于现有的方法.
科学领域:
- 计算机视觉 计算机视觉
- 自然语言处理自然语言处理.
- 人工智能的人工智能
背景情况:
- 弱监督的时间语言接地 (TLG) 旨在使用没有明确时间标签的文本查询来识别视频事件.
- 当前的TLG方法通常依赖于表面的相关性,导致由于缺乏语义连贯性而导致不准确的事件本地化.
- 需要强大的方法来捕捉视频和文本模式的面向事件的语义一致性至关重要.
研究的目的:
- 开发一个新的网络,即以事件为导向的国家调整网络 (ESAN),以改善弱监督的时间语言接地.
- 为了提高模式内的语义连贯性和跨模式的一致性,以便更准确地定位事件.
- 解决现有方法的局限性,这些方法存在部分框架相关性和误导性结果.
主要方法:
- 为文本查询和视频数据构建了"开始-事件-结束"语义状态集.
- 通过从预先训练的大型模型中提取知识来实现交叉模式对齐,采用相对.
- 利用视觉语言模型用于静态框架语义和大型语言模型用于动态语义变化.
主要成果:
- 在两个时间语言基础的基准数据集上,ESAN显著优于现有方法.
- 拟议的方法证明了虚假高相关性减少,从而改善了整体性能.
- 在基于自然语言查询的未经修剪的视频中定位事件时,实现了更高的精度和可靠性.
结论:
- 面向事件的州调整网络 (ESAN) 有效地解决了以前的TLG方法的局限性.
- 由于ESAN能够捕捉以事件为导向的语义连贯性和跨模式一致性,因此在该领域取得了重大进展.
- 这些发现强调了ESAN在更准确,更可靠的时间语言接地方面的潜力.
相关概念视频
Constraints and Statical Determinacy
585
In structural engineering, the equilibrium of a system is not only determined by its equations of equilibrium but also with the help of constraints. Constraints refer to restrictions on the motion of a system. The proper combinations of constraints can minimize the total number of constraints needed to maintain a system in mechanical equilibrium. When this happens, the system is said to be statically determinate. For such systems, the unknown reaction supports can be estimated using equilibrium...
585
The Anchoring-and-Adjustment Heuristic
7.2K
In order to make good decisions, we use our knowledge and our reasoning. Often, this knowledge and reasoning is sound and solid. However, sometimes, we are swayed by biases or by others manipulating a situation. For example, let’s say you and three friends wanted to rent a house and had a combined target budget of $1,600. The realtor shows you only very run-down houses for $1,600 and then shows you a very nice house for $2,000. Might you ask each person to pay more in rent to get the...
7.2K
Associative Learning
309
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
309
Structural Classification of Joints
3.2K
Joints, also known as articulations, are classified based on their structural characteristics, i.e., based on whether the articulating surfaces of the adjacent bones are directly connected by fibrous connective tissue or cartilage, or whether the articulating surfaces contact each other within a fluid-filled joint cavity. These differences serve to divide the joints of the body into three structural classifications.
A fibrous joint is where the adjacent bones are united by fibrous connective...
A fibrous joint is where the adjacent bones are united by fibrous connective...
3.2K
Per-Unit Sequence Models
73
An ideal Y-Y transformer, grounded through neutral impedances, displays per-unit sequence networks akin to those of a single-phase ideal transformer when subjected to balanced positive- or negative-sequence currents. These currents do not produce neutral currents, and their associated voltage drops.
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
73
Statically Indeterminate Problem Solving
369
Statically indeterminate problems are those where statics alone can not determine the internal forces or reactions. Consider a structure comprising two cylindrical rods made of steel and brass. These rods are joined at point B and restrained by rigid supports at points A and C. Now, the reactions at points A and C and the deflection at point B are to be determined. This rod structure is classified as statically indeterminate as the structure has more supports than are necessary for maintaining...
369


