在视频语义细分中的静态动态类级感知一致性
Zhigang Cen1, Ningyan Guo1, Wenjing Xu1
1Beijing University of Posts and Telecommunications, Beijing, 100876, China.
概括
本研究引入了视频语义细分 (VSS) 的新框架,该框架通过专注于类级上下文来改善时间信息的使用. 这种新的方法提高了动态场景中的细分精度.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 视频语义细分 (VSS) 对于自动驾驶和监控等应用至关重要.
- 利用时间信息仍然是VSS的一个关键挑战.
- 之前的方法通常集中在像素级上下文,限制性能.
研究的目的:
- 为视频语义细分 (VSS) 提出一个新的框架,以应对时间信息整合的挑战.
- 在类层面重新思考静态动态语境,以改善细分.
- 为VSS引入一个计算高效的注意力机制.
主要方法:
- 提出了一个静态动态类级感知一致性 (SD-CPC) 框架.
- 引入了多变量类原型,用于类级约束的对比学习.
- 开发了一个静态动态语义对齐模块,使用多尺度和多层次的相关性.
- 实施了基于窗口的注意力图计算方法,以减少计算成本.
主要成果:
- 在VSPW数据集 (MiT-B5) 上实现了51.1mIoU.
- 在城市景观上获得了81.6mIoU,CamVid数据集 (ResNet101) 获得了78.2mIoU.
- 在基准数据集上表现优于现有的最先进方法.
结论:
- 拟议的SD-CPC框架有效地利用类级的时间信息来进行高级视频语义细分.
- 对于类级上下文和语义对齐的新方法显著提高了细分精度.
- 有效的注意力机制有助于VSS的实际应用.
相关概念视频
Perceptual Constancy
532
Perceptual constancy is the ability to recognize that objects remain consistent and unchanged even when their appearance varies due to changes in sensory input. There are four main types of perceptual constancy: size constancy, shape constancy, color constancy, and brightness constancy.
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
532
Force Classification
1.6K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.6K
Gestalt Principles of Perception
439
Gestalt principles provide a framework for understanding how humans perceive objects as unified wholes within their context. These principles are essential in explaining the cognitive processes that make sense of complex visual stimuli by organizing them into coherent groups. One fundamental principle is proximity, which posits that objects located close to each other are perceived as a collective group. For instance, when dots are positioned near one another, the visual system interprets them...
439
Depth Perception and Spatial Vision
909
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
909
Visual System
686
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
686
Perception
573
Perception is a fundamental psychological process that enables individuals to organize, interpret, and consciously experience sensory information. This process is crucial for understanding and interacting with the world around us. It includes both bottom-up and top-down processing, each playing a distinct role in how we perceive our environment.
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...
573


