剪贴:以主题驱动的视频编辑与注意力控制.
Zhichao Zuo1, Zhao Zhang2, Yan Luo1
1School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China.
概括
本研究介绍了Cut-and-Paste,这是一个用于语义视频编辑的新框架. 它使用文本提示符和参考图像进行精确的对象编辑,同时保持背景和时空一致性.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 多媒体处理处理.
背景情况:
- 基于文本的视频编辑提供了多样化的生成,但缺乏对对象细节和编辑区域的精细控制.
- 在视频编辑任务中,对于精确的语义控制通常需要繁的文本描述.
研究的目的:
- 开发一个主题驱动的视频编辑框架,以精确控制编辑区域和背景保存.
- 为了在视频中使用附加参考图像与文本提示一起实现细粒度的语义生成.
- 为了简化视频编辑过程,避免为详细的对象外观提供冗长的文本描述.
主要方法:
- 引入了一个名为Cut-and-Paste的新框架,用于现实世界的语义视频编辑.
- 集成了一个参考图像作为文本驱动视频编辑的补充输入,用于主体驱动的控制.
- 从图像编辑到视频编辑,通过融合相邻的注意力图来扩展交叉注意力控制.
- 在编辑的视频中实现了背景保存和时空一致性之间的平衡.
主要成果:
- 切割和粘贴框架允许精确"切割"源对象和"粘贴"目标对象,以参考图像为指导.
- 与先前技术相比,在需要文本提示和参考图像的视频编辑任务中表现出良好的性能.
- 定量和主观评估证实了拟议方法的有效性.
结论:
- 剪贴和粘贴框架显著提高了语义视频编辑中的控制和精度.
- 使用参考图像与文本提示一起简化了对象的详细编辑,并改善了背景保存.
- 该方法为主题驱动的视频编辑提供了一个实用和有效的解决方案,具有更好的时空一致性.
更多相关视频
相关概念视频
01:16Focusing of Light in the Eye
2.5K
Light rays enter the eye through the cornea, a transparent dome-shaped tissue that is the eye's outermost layer. The cornea bends or refracts, light rays traveling to the pupil. The shape of the cornea determines how much of the light is bent and whether the image will be focused correctly on the retina at the back of the eye. Once the light has passed through both refraction layers, it converges into a single focal point onto a small area. This is where photoreceptors start transforming...
2.5K
02:19Buffer Effectiveness
48.7K
Buffer solutions do not have an unlimited capacity to keep the pH relatively constant . Instead, the ability of a buffer solution to resist changes in pH relies on the presence of appreciable amounts of its conjugate weak acid-base pair. When enough strong acid or base is added to substantially lower the concentration of either member of the buffer pair, the buffering action within the solution is compromised.
The buffer capacity is the amount of acid or base that can be added to a given volume...
The buffer capacity is the amount of acid or base that can be added to a given volume...
48.7K
01:22Purposive Learning
101
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
101
01:25The Anchoring-and-Adjustment Heuristic
7.2K
In order to make good decisions, we use our knowledge and our reasoning. Often, this knowledge and reasoning is sound and solid. However, sometimes, we are swayed by biases or by others manipulating a situation. For example, let’s say you and three friends wanted to rent a house and had a combined target budget of $1,600. The realtor shows you only very run-down houses for $1,600 and then shows you a very nice house for $2,000. Might you ask each person to pay more in rent to get the...
7.2K
01:21Cognitive Learning
222
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
222