语义辅助对象聚类用于多模式引用视频分割
IEEE transactions on pattern analysis and machine intelligence
|September 29, 2025
概括
本研究介绍了语义辅助对象集群网络 (SOC++) 用于多模式引用视频分割. 它通过统一时间交互和交叉模式对齐来提高对象识别和细分精度,以更好地理解视频.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 引用视频细分需要识别由语言线索指定的对象.
- 现有的方法因遮蔽或动作模糊而扎于框架间关系和杂的视觉数据.
- 时间建模中的香草注意力机制可能会导致混乱的表示.
研究的目的:
- 开发一个优化的模型,用于多模式引用视频细分.
- 增强使用框架间关系和全球视频内容的利用.
- 解决以前处理杂视觉嵌入和时间变化的方法的局限性.
主要方法:
- 引入了语义辅助对象集群网络 (SOC) 和其改进版本SOC++.
- 统一的时间选择性交互和交叉模式对齐,以实现视频层次的理解.
- 采用代理辅助的多模式融合,语义集成与渐进的到视频结构,以及多模式查询对比监督.
- 通过强调信息框架和动态查询融合模块,集成了倾向性视频聚合.
主要成果:
- 在受欢迎的参考视频细分基准上,SOC++显著优于最先进的竞争对手.
- 该方法显示了增强的分段稳定性和适应性,特别是对于具有时间变化的文本表达式.
- 在所有测试的基准中实现了显著的绩效利.
结论:
- 拟议的SOC++模型有效地解决了多模式引用视频细分方面的挑战.
- 对时间连贯性和跨模式对齐的强调导致了优越的视频层次理解.
- 该方法提供了一个可靠的解决方案,以语言描述为指导的准确和稳定的对象细分.
相关概念视频
01:29Aggregates Classification
970
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
970
01:22Chunking and Rehearsal in Sensory Memory
561
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
561
01:20Structural Classification of Joints
7.0K
Joints, also known as articulations, are classified based on their structural characteristics, i.e., based on whether the articulating surfaces of the adjacent bones are directly connected by fibrous connective tissue or cartilage, or whether the articulating surfaces contact each other within a fluid-filled joint cavity. These differences serve to divide the joints of the body into three structural classifications.
A fibrous joint is where the adjacent bones are united by fibrous connective...
A fibrous joint is where the adjacent bones are united by fibrous connective...
7.0K