CAFusion:一个渐进的ConvMixer网络,用于上下文感知的红外和可见图像融合
Hafiz Tayyab Mustafa1,2, Hamza Mustafa3, Hassan Alhuzali4
1School of Computer Science and Technology, Zhejiang Normal University, Jinhua, Zhejiang, China.
PloS one
|January 8, 2026
概括
新的深度学习框架CAFusion增强了可见和红外图像的融合. 它通过使用一种新的上下文意识的ConvMixer块和渐进的融合策略,实现了卓越的融合质量和计算效率.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 图像处理 图像处理
背景情况:
- 图像融合的深度学习 (DL) 通常涉及复杂的模型,导致高计算成本和信息丢失.
- 现有的方法可能难以有效地捕获多个规模的上下文信息.
研究的目的:
- 引入CAFusion,一种用于可见 (VI) 和红外 (IR) 图像融合的新型DL框架.
- 与现有方法相比,提高融合质量和计算效率.
主要方法:
- 开发了一个具有上下文意识的ConvMixer块,集成了扩展和深度可分离的卷积.
- 实施了基于注意力的多层次渐进融合战略.
- 使用分层的多尺度解码器进行图像重建.
主要成果:
- 在最先进的DL和基于变压器的方法上,CAFusion表现出优越的性能.
- 在TNO数据集上获得了0.769的SSIM得分,比最好的竞争对手提高了2.07%.
- 展示了增强的融合质量和计算效率.
结论:
- CAFusion有效地将VI和IR图像与高质量和效率融合在一起.
- 拟议的环境意识的ConvMixer块和融合战略是框架成功的关键.
- 对于先进的图像融合任务,CAFusion提供了一个有前途的替代方案.
相关概念视频
01:09Infrared (IR) Spectroscopy: Overview
4.6K
When electromagnetic radiation passes through a material, atoms or molecules transition from a lower to a higher energy state by absorbing radiation corresponding to the energy difference between the two states. The absorption of infrared (IR) radiation causes transitions between vibrational energy levels in a molecule. Therefore, IR spectroscopy is a useful analytical tool for determining the molecular structure of molecules.
Different compounds display unique properties due to their...
Different compounds display unique properties due to their...
4.6K
01:03IR Frequency Region: Fingerprint Region
1.8K
IR spectra are divided into two main regions: the diagnostic region and the fingerprint region. The diagnostic region of the spectrum lies above 1500 cm−1. The absorptions resulting from single-bond vibrations of the N–H, C–H, and O–H stretch at higher wavenumbers and appear on the left side of the spectrum. The stretching absorptions of the C≡C and C≡N occur between 2100–2300 cm−1. In contrast, those arising from stretching absorptions of the...
1.8K
01:20Deconvolution
537
Deconvolution, also known as inverse filtering, is the process of extracting the impulse response from known input and output signals. This technique is vital in scenarios where the system's characteristics are unknown, and they must be inferred from the observable signals.
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
537
02:16Light Acquisition
9.4K
In order to produce glucose, plants need to capture sufficient light energy. Many modern plants have evolved leaves specialized for light acquisition. Leaves can be only millimeters in width or tens of meters wide, depending on the environment. Due to competition for sunlight, evolution has driven the evolution of increasingly larger leaves and taller plants, to avoid shading by their neighbors with contaminant elaboration of root architecture and mechanisms to transport water and nutrients.
9.4K
01:24Vision
59.3K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
59.3K
01:17Convolution Properties II
572
The important convolution properties include width, area, differentiation, and integration properties.
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
The width property indicates that if the durations of input signals are T1 and T2, then the width of the output response equals the sum of both durations, irrespective of the shapes of the two functions. For instance, convolving two rectangular pulses with durations of 2 seconds and 1 second results in a function with a width of 3 seconds.
The area property asserts that the area under the...
572