DTFusion:基于密集的残余PConv-ConvNeXt和纹理对比补偿的红外和可见图像融合
Xinzhi Zhou1, Min He1, Dongming Zhou1
1School of Information, Yunnan University, Kunming 650504, China.
Sensors (Basel, Switzerland)
|January 11, 2024
概括
这项研究介绍了DTFusion,这是一种用于红外和可见图像融合的新型深度学习框架. DTFusion 增强了特征提取和细节补偿,在融合图像质量方面表现优于现有的方法.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 图像处理 图像处理
背景情况:
- 红外和可见图像融合整合了来自两个源图像的互补信息,以创建更具信息性的融合图像.
- 现有的深度学习融合方法经常使用小内核或固定的融合策略,限制特征表示和整体性能.
研究的目的:
- 提出一个新的端到端红外和可见图像融合框架,DTFusion,以克服当前深度学习方法的局限性.
- 通过解决受体场大小和特征表示方面的局限性,增强特征提取能力并提高融合图像的质量.
主要方法:
- 开发了DTFusion,这是一个端到端的框架,包含一个剩余的PConv-ConvNeXt模块 (RPCM) 以实现更大的受体场的高效特征提取.
- 引入了一个纹理对比补偿模块 (TCCM),利用渐变残留物和注意力机制来保存和增强纹理细节和对比度.
- 在编码器中使用密集连接和四个卷积层用于特征重建.
主要成果:
- 与公共数据集上的最先进的融合方法相比,DTFusion在公共数据集上表现出更高的性能.
- 拟议的框架在主观视觉质量和客观绩效指标方面取得了更好的结果.
- 实验结果验证了RPCM和TCCM在改善特征表示和融合质量的有效性.
结论:
- DTFusion为红外和可见图像融合提供了先进的解决方案,显著改善了特征表示和融合性能.
- 新型模块 (RPCM和TCCM) 有效地解决了小受体场的局限性和现有方法中不充分的细节补偿.
- 拟议的框架提供了一种强大而高性能的方法,用于从红外和可见光源生成信息化的融合图像.
相关概念视频
Infrared (IR) Spectroscopy: Overview
1.8K
When electromagnetic radiation passes through a material, atoms or molecules transition from a lower to a higher energy state by absorbing radiation corresponding to the energy difference between the two states. The absorption of infrared (IR) radiation causes transitions between vibrational energy levels in a molecule. Therefore, IR spectroscopy is a useful analytical tool for determining the molecular structure of molecules.
Different compounds display unique properties due to their...
Different compounds display unique properties due to their...
1.8K
IR Frequency Region: Fingerprint Region
900
IR spectra are divided into two main regions: the diagnostic region and the fingerprint region. The diagnostic region of the spectrum lies above 1500 cm−1. The absorptions resulting from single-bond vibrations of the N–H, C–H, and O–H stretch at higher wavenumbers and appear on the left side of the spectrum. The stretching absorptions of the C≡C and C≡N occur between 2100–2300 cm−1. In contrast, those arising from stretching absorptions of the...
900
Convolution Properties II
203
The important convolution properties include width, area, differentiation, and integration properties.
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
203
Deconvolution
160
Deconvolution, also known as inverse filtering, is the process of extracting the impulse response from known input and output signals. This technique is vital in scenarios where the system's characteristics are unknown, and they must be inferred from the observable signals.
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
160
Convolution Properties I
152
Convolution computations can be simplified by utilizing their inherent properties.
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
The commutative property reveals that the input and the impulse response of an LTI (Linear Time-Invariant) system can be interchanged without affecting the output:
152
IR Frequency Region: X–H Stretching
981
In IR spectroscopy, signals produced by the X−H bonds (such as C−H, O−H, or N−H) can be observed in the frequency range of 2700–4000 cm–1. The C−H stretching vibration forms sharp bands in the region 2850–3000 cm–1. The presence of the O−H stretching vibration leads to the forming of an absorption band in the frequency range 3650–3200 cm−1. At the same time, N−H stretching can be confirmed by absorption bands in...
981


