一个基于DINO的渐进式语义增强红外和可见图像融合网络
Shihan Yao1, Zhonghui Pei1, Huiqin Zhang1
1Wuhan Institute of Technology, Wuhan, 430250, China.
概括
这项研究引入了一个新的红外和可见图像融合网络 (DPSEF),该网络使用自主监督学习来提高语义理解. DPSEF网络产生高质量的融合图像,具有丰富的细节和语义信息,用于更好的下游应用.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 图像处理 图像处理
背景情况:
- 现有的红外和可见图像融合方法往往忽略了对于下游任务至关重要的语义信息.
- 目前的语义驱动方法是依赖于具有有限语义目标的标记数据所限制的.
研究的目的:
- 提出一个新的基于DINO的渐进语义增强红外和可见图像融合网络 (DPSEF).
- 为了利用自主监督学习来增强语义特征提取和整合图像融合.
主要方法:
- 使用DINO (自我监督模型) 来从未标记的图像中提取细粒度的空间语义特征.
- 引入一个语义增强融合模块 (SEFM),逐步将语义先验注入到融合网络中.
- 开发一个渐进的融合机制,以引导该模型向目标相关地区.
主要成果:
- 在融合图像视觉质量方面,DPSEF显著超过主流算法.
- 通过定性和定量分析,证明了高水平视觉应用的巨大潜力.
- 在多焦点图像融合任务上验证了网络的通用性和稳定性.
结论:
- 拟议的DPSEF网络有效地整合了丰富的语义和详细信息,以实现高质量的图像融合.
- DPSEF通过利用未标记的数据进行语义增强来解决现有方法的局限性.
- 该网络显示出对推进计算机视觉应用程序,需要语义丰富的融合图像的显著前景.
相关概念视频
01:15Insensitive Nuclei Enhanced by Polarization Transfer (INEPT)
982
Insensitive Nuclei Enhanced by Polarization Transfer (INEPT) is an advanced Nuclear Magnetic Resonance (NMR) technique specifically designed to detect and enhance the signals of low-abundance nuclei, such as carbon-13 and nitrogen-15, in small molecules. The fundamental principle behind INEPT is the transfer of polarization from a more abundant and highly polarizable nucleus, typically hydrogen-1, to the low-abundance nucleus of interest. This process effectively boosts the NMR signal of the...
982
01:09Infrared (IR) Spectroscopy: Overview
4.6K
When electromagnetic radiation passes through a material, atoms or molecules transition from a lower to a higher energy state by absorbing radiation corresponding to the energy difference between the two states. The absorption of infrared (IR) radiation causes transitions between vibrational energy levels in a molecule. Therefore, IR spectroscopy is a useful analytical tool for determining the molecular structure of molecules.
Different compounds display unique properties due to their...
Different compounds display unique properties due to their...
4.6K
02:16Light Acquisition
9.3K
In order to produce glucose, plants need to capture sufficient light energy. Many modern plants have evolved leaves specialized for light acquisition. Leaves can be only millimeters in width or tens of meters wide, depending on the environment. Due to competition for sunlight, evolution has driven the evolution of increasingly larger leaves and taller plants, to avoid shading by their neighbors with contaminant elaboration of root architecture and mechanisms to transport water and nutrients.
9.3K
01:37Super-resolution Fluorescence Microscopy
12.2K
Super-resolution fluorescence microscopy (SRFM) provides a better resolution than conventional fluorescence microscopy by reducing the point spread function (PSF). PSF is the light intensity distribution from a point that causes it to appear blurred. Due to PSF, each fluorescing point appears bigger than its actual size, and it is the PSF interference of nearby fluorophores that causes the blurred image. Various approaches to achieving higher resolution through SRFM have recently been...
12.2K
01:15Depth Perception and Spatial Vision
1.8K
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
1.8K
01:24Vision
59.3K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
59.3K