谁,什么,在哪里:复合语义实例搜索故事视频
概括
本研究介绍了一种新的部分分解 (PD) 方法,用于3W (谁,什么,在哪里) 视频实例搜索,通过建模语义相关性来提高准确性. 在故事视频中,PD方法的性能优于现有的完全分解和非分解方法.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 多媒体检索检索
背景情况:
- 准确识别谁,什么和哪里 (3W) 语义对于理解视频叙述至关重要.
- 现有的3W视频实例搜索 (INS) 方法,完全分解 (CD) 和非分解 (ND),在处理复合语义和细粒度对象识别方面存在局限性.
研究的目的:
- 为3W复合语义视频实例搜索问题提出一种新的视频结构感知部分分解 (PD) 方法.
- 通过模拟视频结构中的语义相关性来解决现有方法的不准确性和不完整性.
主要方法:
- 拟议的PD方法部分将3W INS问题分解为三个相关的2W INS子问题:人-动作,动作-位置和位置-人.
- 语义对之间的相关性在,镜头和场景层面上建模,利用空间一致性和时间连续性.
- 该方法利用故事视频的层次结构来实现身份匹配,逻辑一致和内容一致的结果.
主要成果:
- 在三个大规模的3W INS数据集上进行了广泛的实验,这些数据集是从电视剧 (Eastenders,Friends,The Big Bang Theory) 中创建的,总共超过67万张视频镜头.
- 与最先进的CD和ND方法相比,PD方法在故事视频中为3W INS表现出了更高的性能.
- 拟议的方法有效地捕捉了对准确的视频检索至关重要的语义相关性.
结论:
- 视频结构意识的部分分解 (PD) 方法在3W复合语义视频实例搜索中提供了显著的进步.
- PD有效地模拟故事视频中的语义相关性,从而获得更准确和更连贯的检索结果.
- 这项工作为复杂的视频理解和检索任务提供了强大的框架.
相关概念视频
Stereotype Content Model
13.9K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
13.9K
Semiconductors
520
There is variation in the electrical conductivity of materials - metals, semiconductors, and insulators that are showcased with the help of the energy band diagrams.
Metals such as copper (Cu), zinc (Zn), or lead (Pb) have low resistivity and feature conduction bands that are either not fully occupied or overlap with the valence band, making a bandgap non-existent. This allows electrons in the highest energy levels of the valence band to easily transition to the conduction band upon gaining...
Metals such as copper (Cu), zinc (Zn), or lead (Pb) have low resistivity and feature conduction bands that are either not fully occupied or overlap with the valence band, making a bandgap non-existent. This allows electrons in the highest energy levels of the valence band to easily transition to the conduction band upon gaining...
520
Schemata
58
A schema is a mental construct that organizes related concepts, allowing the brain to process information efficiently. Upon activation, schemata facilitate assumptions about people or objects.
Two types of schemata are:
Two types of schemata are:
58
State Space Representation
160
The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
Consider an RLC circuit, a...
160
Types of Semiconductors
476
Intrinsic semiconductors are highly pure materials with no impurities. At absolute zero, these semiconductors behave as perfect insulators because all the valence electrons are bound, and the conduction band is empty, disallowing electrical conduction. The Fermi level is a concept used to describe the probability of occupancy of energy levels by electrons at thermal equilibrium. In intrinsic semiconductors, the Fermi level is positioned at the midpoint of the energy gap at absolute zero. When...
476
Case Studies
11.6K
There are many research methods available to psychologists in their efforts to understand, describe, and explain behavior and the cognitive and biological processes that underlie it.
11.6K


