基于深度学习的红外和可见光图像融合的可解释性分析
Bo Yuan1, Hongyu Sun2, YinJing Guo1
1Shandong University of Science and Technology, Qingdao, 266590, China.
Scientific reports
|January 17, 2025
概括
这项研究提高了可见光和红外图像融合的解释性,使用了新的CNN-Transformer模型和DDFImage算法. 这种方法提高了模型的可信性和图像处理中的实际应用.
科学领域:
- 机器学习 机器学习
- 图像处理 图像处理
- 计算机视觉 计算机视觉
背景情况:
- 可解释性对于机器学习和图像处理中可靠的人工智能至关重要.
- 可见光和红外图像融合提供了更好的场景理解,但往往缺乏透明度.
- 目前的可解释性方法可能无法完全捕捉多式联络融合模型的细微差别.
研究的目的:
- 研究和提高可见光和红外图像融合技术的解释性.
- 提高多式联动图像融合模型的可信度和实际可用性.
- 为融合模型的内部运作提供更深入的见解.
主要方法:
- 提出了一个多式图像融合模型,集成卷积神经网络 (CNN) 来进行本地特征提取和变压器的全球注意力机制.
- 采用Delta调试试管图像 (DDFImage) 算法来生成模型透明度的本地解释信息.
- 对生成的解释性融合图像进行特征重要性分析,以了解模型决策.
主要成果:
- 拟议的DDFImage算法显著提高了图像融合模型的可解释性.
- 功能重要性分析揭示了模型对融合图像的内部处理的关键见解.
- 对比分析表明,拟议的可解释性方法的性能优于现有的算法.
结论:
- 开发的方法成功地提高了可见和红外图像融合模型的解释性.
- 增强的模型透明度增加了对其理解和应用的信心.
- 这项工作为可解释AI在图像融合任务中的实际部署提供了有价值的参考资料.
相关概念视频
01:09Infrared (IR) Spectroscopy: Overview
1.4K
When electromagnetic radiation passes through a material, atoms or molecules transition from a lower to a higher energy state by absorbing radiation corresponding to the energy difference between the two states. The absorption of infrared (IR) radiation causes transitions between vibrational energy levels in a molecule. Therefore, IR spectroscopy is a useful analytical tool for determining the molecular structure of molecules.
Different compounds display unique properties due to their...
Different compounds display unique properties due to their...
1.4K
01:20Deconvolution
129
Deconvolution, also known as inverse filtering, is the process of extracting the impulse response from known input and output signals. This technique is vital in scenarios where the system's characteristics are unknown, and they must be inferred from the observable signals.
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
129
01:03IR Frequency Region: Fingerprint Region
739
IR spectra are divided into two main regions: the diagnostic region and the fingerprint region. The diagnostic region of the spectrum lies above 1500 cm−1. The absorptions resulting from single-bond vibrations of the N–H, C–H, and O–H stretch at higher wavenumbers and appear on the left side of the spectrum. The stretching absorptions of the C≡C and C≡N occur between 2100–2300 cm−1. In contrast, those arising from stretching absorptions of the...
739
01:02Flame Photometry: Overview
446
Flame photometry, also known as flame emission spectrometry, is a technique used for the qualitative and quantitative analysis of elements present in a sample using a flame as the source of excitation energy. The concept of flame photometry was realized in the early 1860s by Kirchhoff and Bunsen, who discovered that specific elements emit characteristic radiation when excited in flames. The first instrument developed for this purpose was used to measure sodium (Na) in plant ash using a Bunsen...
446
01:22Force Classification
1.1K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.1K
01:19IR Spectrum
916
When infrared (IR) radiation passes through a molecule, the bonds stretch or bend by absorbing the radiation. This absorption creates the molecule's absorption spectrum, which is the plot of its percentage transmittance versus wavenumber.
Transmittance is defined as the ratio of the radiant power passing through a sample to that from the radiation's source. Multiplying the transmittance by 100 gives the percent transmittance (%T), which varies between 100% (no absorption) and 0%...
Transmittance is defined as the ratio of the radiant power passing through a sample to that from the radiation's source. Multiplying the transmittance by 100 gives the percent transmittance (%T), which varies between 100% (no absorption) and 0%...
916