MJ-GAN:具有多粒度特征提取和联合注意力融合的生成对抗网络,用于红外和可见图像融合
Danqing Yang1, Xiaorui Wang1, Naibo Zhu2
1School of Optoelectronic Engineering, Xidian University, Xi'an 710071, China.
Sensors (Basel, Switzerland)
|July 29, 2023
概括
本研究引入了一种新的多尺度特征提取和联合注意力融合 (MJ-GAN) 方法,用于红外和可见图像融合 (IVIF). 这种方法增强了融合图像的纹理和对比度,优于现有的算法.
科学领域:
- 计算机视觉 计算机视觉
- 图像处理 图像处理
- 人工智能的人工智能
背景情况:
- 红外和可见图像融合 (IVIF) 面临着从可见图像中提取和融合丰富的纹理和红外图像的对比度的挑战.
- 由于手工操作和单一尺度的特征提取,现有的核聚变方法扎,导致信息不足和目标可见性差.
研究的目的:
- 为IVIF开发一个端到端的方法,克服现有方法的局限性.
- 改进多粒度特征的提取和融合,以增强融合图像中的纹理和对比度.
主要方法:
- 提出了一个基于多级特征提取 (MFE) 和联合注意力融合 (JAF) 的生成对抗网络 (MJ-GAN) 框架.
- 具有自我注意力的MFE模块嵌入到双流发生器中,以提取和完善多粒度特征.
- 一个JAF网络融合了突出特征,一个双分辨器增强了最终的融合图像的纹理和对比度.
主要成果:
- 拟议的MJ-GAN方法有效地提取了全面的多粒度特征.
- JAF网络成功地重新校准和合并功能,以改善图像重建.
- 实验表明,与四个数据集上的九个领先算法相比,视觉质量和定量性能优越.
结论:
- MJ-GAN框架为IVIF提供了一个强大的解决方案,解决了特征提取和融合方面的局限性.
- 该方法显著增强了纹理和对比度,从而产生了优越的融合图像质量.
- 这种方法代表了红外和可见图像融合技术的重大进步.
相关概念视频
01:09Infrared (IR) Spectroscopy: Overview
1.9K
When electromagnetic radiation passes through a material, atoms or molecules transition from a lower to a higher energy state by absorbing radiation corresponding to the energy difference between the two states. The absorption of infrared (IR) radiation causes transitions between vibrational energy levels in a molecule. Therefore, IR spectroscopy is a useful analytical tool for determining the molecular structure of molecules.
Different compounds display unique properties due to their...
Different compounds display unique properties due to their...
1.9K
01:13Attenuated Total Reflectance (ATR) Infrared Spectroscopy: Overview
418
Attenuated total reflectance (ATR) infrared spectroscopy is a powerful analytical technique used to study the composition of materials. It is widely employed in chemistry, materials science, forensic science, and other fields where sample characterization is required. ATR has several advantages over traditional transmission IR spectroscopy, including the requirement of little to no sample preparation and the ability to analyze a wide range of samples.
The ATR process begins by directing a beam...
The ATR process begins by directing a beam...
418
02:16Light Acquisition
8.5K
In order to produce glucose, plants need to capture sufficient light energy. Many modern plants have evolved leaves specialized for light acquisition. Leaves can be only millimeters in width or tens of meters wide, depending on the environment. Due to competition for sunlight, evolution has driven the evolution of increasingly larger leaves and taller plants, to avoid shading by their neighbors with contaminant elaboration of root architecture and mechanisms to transport water and nutrients.
8.5K
01:21IR Absorption Frequency: Hybridization
712
Hydrocarbons such as alkanes, alkenes, and alkynes show characteristic C–H stretching absorption bands. These IR stretching frequencies depend on the hybridization of the involved carbon atom and can be explained in terms of the s character of each hybridized atomic orbital.
Among the sp, sp2, and sp3 hybridized orbitals, sp orbitals have the maximum s character (50%). Consequently, the electrons are held more closely to the nucleus, resulting in stronger and shorter C–H bonds that...
Among the sp, sp2, and sp3 hybridized orbitals, sp orbitals have the maximum s character (50%). Consequently, the electrons are held more closely to the nucleus, resulting in stronger and shorter C–H bonds that...
712
01:03IR Frequency Region: Fingerprint Region
940
IR spectra are divided into two main regions: the diagnostic region and the fingerprint region. The diagnostic region of the spectrum lies above 1500 cm−1. The absorptions resulting from single-bond vibrations of the N–H, C–H, and O–H stretch at higher wavenumbers and appear on the left side of the spectrum. The stretching absorptions of the C≡C and C≡N occur between 2100–2300 cm−1. In contrast, those arising from stretching absorptions of the...
940
01:16Focusing of Light in the Eye
2.9K
Light rays enter the eye through the cornea, a transparent dome-shaped tissue that is the eye's outermost layer. The cornea bends or refracts, light rays traveling to the pupil. The shape of the cornea determines how much of the light is bent and whether the image will be focused correctly on the retina at the back of the eye. Once the light has passed through both refraction layers, it converges into a single focal point onto a small area. This is where photoreceptors start transforming...
2.9K