无监督的低光视频增强与时空共注意力变压器
概括
本研究介绍了LightenFormer,这是第一个用于低光视频增强的无监督方法. 它使用时空变压器来提高亮度和时间一致性,克服监督卷积神经网络 (CNN) 的局限性.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 图像处理 图像处理
背景情况:
- 监督卷积神经网络 (CNN) 主导低光视频增强,但由于合成训练,与真实世界的数据斗争.
- 现有的方法表现出时间不一致性,如闪和运动模糊,特别是在大型运动中,由于CNN对远程依赖性的有限感知.
研究的目的:
- 开发第一个无监督的低光视频增强方法.
- 为了提高视频亮度,同时保持时间一致性,即使是大动作.
- 为了克服在合成数据上训练的监督方法的泛化局限性.
主要方法:
- 提出LightenFormer,一种无人监督的低光视频增强方法,使用时空协同注意力变压器.
- 引入了用于自适应动态范围调整的S曲线估计网络 (SCENet).
- 开发了空间时间精细化网络 (STRNet),采用了新的空间时间协同注意力变压器 (STCAT) 进行时间一致性和远程依赖性建模.
- 为无监督训练设计了两个非参考损失函数,利用S曲线可逆性和噪声独立性.
主要成果:
- LightenFormer有效地提高了亮度,并在低光视频中保持时间一致性.
- 时空共同注意力变压器捕捉了远程的空间和时间相关性,以改进动作处理.
- 在广泛的实验中,在SDSD和LLIV-Phone数据集上超越了最先进的方法.
结论:
- LightenFormer为低光视频增强提供了一种新的无监督方法,解决了以前监督方法的关键局限性.
- 该方法在亮度增强和时间稳定方面表现出卓越的性能.
- 在现实场景中为更强大和更普遍的低光视频增强技术铺平了道路.
相关概念视频
01:26Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K
02:16Light Acquisition
8.5K
In order to produce glucose, plants need to capture sufficient light energy. Many modern plants have evolved leaves specialized for light acquisition. Leaves can be only millimeters in width or tens of meters wide, depending on the environment. Due to competition for sunlight, evolution has driven the evolution of increasingly larger leaves and taller plants, to avoid shading by their neighbors with contaminant elaboration of root architecture and mechanisms to transport water and nutrients.
8.5K
01:20Deconvolution
188
Deconvolution, also known as inverse filtering, is the process of extracting the impulse response from known input and output signals. This technique is vital in scenarios where the system's characteristics are unknown, and they must be inferred from the observable signals.
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
188
01:25Transformers with Off-Nominal Turns Ratios
176
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
176
01:16Types Of Transformers
1.0K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.0K
01:15Depth Perception and Spatial Vision
720
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
720