不统一的曝光成像通过神经形态的快门控制
概括
这项研究引入了一个神经形态快门控制 (NSC) 系统,使用基于事件的摄像头来适应性地管理曝光,减少运动模糊和噪音. 自主监督的除网络 (SEID) 在具有挑战性的现实环境中进一步提高了图像质量.
科学领域:
- 计算机视觉 计算机视觉
- 神经形态工程的神经形态工程
- 图像处理 图像处理
背景情况:
- 传统相机在适应式快门控制方面扎着实时运动感知,限制了恶劣环境中的成像.
- 不均的曝光提供了灵活性,但在管理运动模糊和噪音方面带来了挑战.
- 现有的方法缺乏感知框架内动态信息的能力,以适应性暴露控制.
研究的目的:
- 开发一种新的神经形态快门控制 (NSC) 系统,用于实时自适应相机快门控制.
- 通过利用基于事件的传感来克服传统摄像机在动态环境中的局限性.
- 通过适应性曝光和先进的无声化,通过减轻运动模糊和噪声来提高图像质量.
主要方法:
- 提出了一个神经形态快门控制 (NSC) 系统,利用基于事件的传感器的低延迟进行实时运动监控和适应性曝光.
- 开发了一种基于事件的图像否定网络 (SEID),使用自主监督学习来稳定信号对噪声比 (SNR) 和否定图像.
- 从事件中探索图像噪声统计和跨运动信息,以生成SEID的自我监控信号.
- 在混合摄像头原型中实现了NSC系统,并收集了同步的真实世界数据集.
主要成果:
- 在动态场景中,NSC系统有效地避免了运动模糊,并减轻了动态场景中的即时噪声.
- SEID网络成功地稳定了因异常曝光时间造成的不一致的SNR.
- 合成和现实世界数据集的实验表明,与最先进的方法相比,性能优越.
- 混合摄像头原型验证了该系统在各种现实世界的场景中的有效性.
结论:
- 拟议的神经形态快门控制 (NSC) 系统为在具有挑战性的环境中进行自适应成像提供了强大的解决方案.
- 基于事件的传感与自我监督的学习相结合,可以通过实时运动适应获得高质量的图像.
- 开发的系统显著提升了适应式摄像机快门控制在实际应用中的功能.
更多相关视频
相关概念视频
01:37Neural Regulation
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
01:24Vision
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
01:21G-Protein Gated Ion Channels
GPCRs are primarily responsible for our sense of smell, taste, and vision. The binding of a sensory stimulus activates GPCR to stimulate effector proteins, many of which are ion channels in the sensory organs. GPCRs modulate the opening and closing of the target ion channels either directly by binding them, or by releasing second messengers that activate these channels. As ions move across the membrane, the membrane potential is altered, which induces an appropriate response.
Sensory organs,...
Sensory organs,...
01:22Photoreceptors and Visual Pathways
At the molecular level, visual signals trigger transformations in photopigment molecules, resulting in changes in the photoreceptor cell's membrane potential. The photon's energy level is denoted by its wavelength, with each specific wavelength of visible light associated with a distinct color. The spectral range of visible light, classified as electromagnetic radiation, spans from 380 to 720 nm. Electromagnetic radiation wavelengths exceeding 720 nm fall under the infrared category, whereas...
01:26Visual System
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...