事件HDR:从事件到高速的HDR视频和超越
IEEE transactions on pattern analysis and machine intelligence
|October 9, 2024
概括
本研究引入了一种用于从事件摄像头数据中重建高速,高动态范围 (HDR) 视频的新方法. 该方法使用循环神经网络和新的真实世界数据集来克服以前的限制.
科学领域:
- 计算机视觉 计算机视觉
- 神经形态工程的神经形态工程
- 视频处理 视频处理
背景情况:
- 事件摄像机提供低延迟和高灵敏度,非常适合动态场景.
- 之前用于事件到HDR视频重建的方法面临着文物和率的挑战.
- 有限的真实世界数据集阻碍了基于事件的HDR视频重建的发展.
研究的目的:
- 开发一种可靠的方法来从事件流中重建高速,高动态范围 (HDR) 视频.
- 创建一个新的,现实世界的数据集,用于事件到HDR视频重建.
- 评估该方法的性能,并探索其在下游计算机视觉任务中的应用.
主要方法:
- 一个反复卷积神经网络 (CNN) 架构被用于视频重建.
- 关键框架指导被纳入以减轻稀疏事件数据的错误积累.
- 一个新的光学系统被设计用于捕获配对的高速HDR视频和事件流,创建一个真实世界的数据集.
主要成果:
- 拟议的方法成功地从事件序列中生成高质量,高速的HDR视频.
- 新收集的数据集是首个用于事件对HDR重建的真实配对数据集,避免模拟不准确.
- 实验结果验证了重复CNN的有效性,使用关键框架指导.
结论:
- 开发的方法推进了事件到HDR视频重建,从而实现高保真性,高速输出.
- 新的真实世界数据集将加速基于事件的视觉研究.
- 该方法对各种HDR计算机视觉应用具有前景,包括对象检测和深度估计.
相关概念视频
Light Acquisition
In order to produce glucose, plants need to capture sufficient light energy. Many modern plants have evolved leaves specialized for light acquisition. Leaves can be only millimeters in width or tens of meters wide, depending on the environment. Due to competition for sunlight, evolution has driven the evolution of increasingly larger leaves and taller plants, to avoid shading by their neighbors with contaminant elaboration of root architecture and mechanisms to transport water and nutrients.
Instantaneous Velocity - II
Instantaneous velocity is the quantity that measures how fast an object is moving along its path. In other words, the instantaneous velocity of an object is the limit of the average velocity as the elapsed time approaches zero, or the derivative of displacement with respect to time. Like average velocity, the instantaneous velocity is a vector with the dimensions of length per unit time. Instantaneous velocity can have both positive and negative values. The instantaneous velocity can be...
Deconvolution
Deconvolution, also known as inverse filtering, is the process of extracting the impulse response from known input and output signals. This technique is vital in scenarios where the system's characteristics are unknown, and they must be inferred from the observable signals.
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Aliasing
Accurate signal sampling and reconstruction are crucial in various signal-processing applications. A time-domain signal's spectrum can be revealed using its Fourier transform. When this signal is sampled at a specific frequency, it results in multiple scaled replicas of the original spectrum in the frequency domain. The spacing of these replicas is determined by the sampling frequency.
If the sampling frequency is below the Nyquist rate, these replicas overlap, preventing the original signal...
If the sampling frequency is below the Nyquist rate, these replicas overlap, preventing the original signal...
Downsampling
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
Upsampling
Managing signal sampling rates is essential in digital signal processing to maintain signal integrity. A decimated signal, characterized by a reduced frequency range due to its lower sampling rate, can be upsampled by inserting zeros between each sample. This upsampling process expands the original spectrum and introduces repeated spectral replicas at intervals dictated by the new Nyquist frequency. To refine this zero-inserted sequence, it is passed through a lowpass filter with a cutoff...


