TDDFusion:用于红外和可见图像融合的目标驱动的双分支网络
Siyu Lu1,2,3, Xiangzhou Ye2,3,4, Junmin Rao2,3,4
1School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China.
Sensors (Basel, Switzerland)
|January 11, 2024
概括
通过对目标物体进行聚焦,TDDFusion增强了红外和可见图像的融合. 这种目标驱动的双分支融合网络提高了下游对象检测性能.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 图像处理 图像处理
背景情况:
- 图像融合将红外和可见图像结合起来,以创建具有互补特征的统一表示.
- 传统的深度学习融合方法往往忽视了下游任务的目标对象的重要性.
- 现有的方法可能无法有效地弥合像素级融合和对象检测之间的性能差距.
研究的目的:
- 引入TDDFusion,一个目标驱动的双分支融合网络,旨在提高合图像中目标对象的突出位置.
- 通过优先考虑对物体检测的任务导向优化来解决当前融合技术的局限性.
- 提高融合图像对后续决策过程的相关性.
主要方法:
- 使用一个并行,双分支的特征提取网络与全球语义变压器 (GST) 和本地纹理编码器 (LTE).
- 在训练过程中集成一个对象检测子模块,以反向传播语义损失以实现面向任务的融合.
- 使用一种新的损失函数,该函数包含目标位置信息,以增强对象特定的对比度和细节.
主要成果:
- 与最先进的方法相比,TDDFusion在保护全球环境信息和当地细节方面表现出优越性.
- 该模型有效地平衡了像素强度,并保持了目标对象的纹理.
- 在下游物体检测任务中观察到显著的优势,优于现有的聚变技术.
结论:
- 在融合的红外和可见图像中,TDDFusion有效地提高了目标物体的可见性.
- 拟议的网络架构和损失函数能够实现面向任务的融合,从而提高对象检测的性能.
- 这种方法为需要准确识别和分析目标对象的图像融合应用提供了卓越的解决方案.
更多相关视频
10:30Simultaneously Capturing Real-time Images in Two Emission Channels Using a Dual Camera Emission Splitting System: Applications to Cell Adhesion
Published on: September 4, 2013
9.6K
12:51Simultaneous Multicolor Imaging of Biological Structures with Fluorescence Photoactivation Localization Microscopy
Published on: December 9, 2013
8.9K
相关概念视频
Infrared (IR) Spectroscopy: Overview
1.8K
When electromagnetic radiation passes through a material, atoms or molecules transition from a lower to a higher energy state by absorbing radiation corresponding to the energy difference between the two states. The absorption of infrared (IR) radiation causes transitions between vibrational energy levels in a molecule. Therefore, IR spectroscopy is a useful analytical tool for determining the molecular structure of molecules.
Different compounds display unique properties due to their...
Different compounds display unique properties due to their...
1.8K
IR Frequency Region: Fingerprint Region
900
IR spectra are divided into two main regions: the diagnostic region and the fingerprint region. The diagnostic region of the spectrum lies above 1500 cm−1. The absorptions resulting from single-bond vibrations of the N–H, C–H, and O–H stretch at higher wavenumbers and appear on the left side of the spectrum. The stretching absorptions of the C≡C and C≡N occur between 2100–2300 cm−1. In contrast, those arising from stretching absorptions of the...
900
Total Internal Reflection Fluorescence Microscopy
5.8K
Total internal reflection fluorescence microscopy or TIRF is an advanced microscopic technique used to visualize fluorophores in samples close to a solid surface with a higher refractive index, such as a glass coverslip. TIRF only allows fluorophores in proximity to the solid surface to be excited. When light from a medium with a lower refractive index (such as air) hits the glass coverslip at a critical angle, the light undergoes total internal reflection stead of passing through the glass.
5.8K
IR Frequency Region: X–H Stretching
981
In IR spectroscopy, signals produced by the X−H bonds (such as C−H, O−H, or N−H) can be observed in the frequency range of 2700–4000 cm–1. The C−H stretching vibration forms sharp bands in the region 2850–3000 cm–1. The presence of the O−H stretching vibration leads to the forming of an absorption band in the frequency range 3650–3200 cm−1. At the same time, N−H stretching can be confirmed by absorption bands in...
981
IR Spectrum
1.0K
When infrared (IR) radiation passes through a molecule, the bonds stretch or bend by absorbing the radiation. This absorption creates the molecule's absorption spectrum, which is the plot of its percentage transmittance versus wavenumber.
Transmittance is defined as the ratio of the radiant power passing through a sample to that from the radiation's source. Multiplying the transmittance by 100 gives the percent transmittance (%T), which varies between 100% (no absorption) and 0%...
Transmittance is defined as the ratio of the radiant power passing through a sample to that from the radiation's source. Multiplying the transmittance by 100 gives the percent transmittance (%T), which varies between 100% (no absorption) and 0%...
1.0K
