STFuse:通过半监督转移学习进行红外和可见图像融合
IEEE transactions on neural networks and learning systems
|November 8, 2023
概括
本研究介绍了STFuse,这是一种半监督的传输学习方法,用于红外和可见图像融合 (IVIF). 它通过将多焦图像融合的知识转移到图像中,有效地融合图像,克服了缺少基本真相数据的局限性.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 图像处理 图像处理
背景情况:
- 红外和可见图像融合 (IVIF) 旨在将源图像中的互补信息结合起来.
- 在没有基本事实或事先知识的情况下,定义补充信息是很困难的.
研究的目的:
- 为IVIF提出一个半监督转移学习方法 (STFuse).
- 为了解决IVIF中缺乏基础真相和事先知识的局限性.
主要方法:
- 使用半监督转移学习将知识从多焦图像融合 (MFIF) 转移到IVIF.
- 使用指导损失来过特定任务的知识,使跨任务学习成为可能.
- 引入了一个跨功能增强模块 (CEM),用于功能改进和集成,具有自我注意和相互注意.
主要成果:
- STFuse有效地缓解了核聚变性能缺乏基本真相的问题.
- 该方法表明与先前的知识相比,优越的互补表达能力.
- 在视觉质量,统计指标和高级视觉任务性能方面取得了显著的优势.
结论:
- 通过跨任务的知识转移,STFuse为IVIF提供了一种新且有效的方法.
- 该方法与最先进的技术相比表现强.
- 强调了半监督转移学习在图像融合任务中的潜力.
相关概念视频
Total Internal Reflection Fluorescence Microscopy
5.8K
Total internal reflection fluorescence microscopy or TIRF is an advanced microscopic technique used to visualize fluorophores in samples close to a solid surface with a higher refractive index, such as a glass coverslip. TIRF only allows fluorophores in proximity to the solid surface to be excited. When light from a medium with a lower refractive index (such as air) hits the glass coverslip at a critical angle, the light undergoes total internal reflection stead of passing through the glass.
5.8K
IR Frequency Region: Fingerprint Region
911
IR spectra are divided into two main regions: the diagnostic region and the fingerprint region. The diagnostic region of the spectrum lies above 1500 cm−1. The absorptions resulting from single-bond vibrations of the N–H, C–H, and O–H stretch at higher wavenumbers and appear on the left side of the spectrum. The stretching absorptions of the C≡C and C≡N occur between 2100–2300 cm−1. In contrast, those arising from stretching absorptions of the...
911
Super-resolution Fluorescence Microscopy
7.0K
Super-resolution fluorescence microscopy (SRFM) provides a better resolution than conventional fluorescence microscopy by reducing the point spread function (PSF). PSF is the light intensity distribution from a point that causes it to appear blurred. Due to PSF, each fluorescing point appears bigger than its actual size, and it is the PSF interference of nearby fluorophores that causes the blurred image. Various approaches to achieving higher resolution through SRFM have recently been...
7.0K
Ultraviolet and Visible (UV–Vis) Spectroscopy: Overview
2.7K
Ultraviolet–visible (UV–visible or UV–Vis) spectroscopy is an analytical technique that investigates the interaction between matter and UV–Vis light within the electromagnetic spectrum. This method is widely used for its versatility, simplicity, and relatively quick data acquisition, making it valuable for both qualitative and quantitative analysis. When UV–Vis radiation passes through a material, molecules absorb light depending on the energy required for...
2.7K
Infrared (IR) Spectroscopy: Overview
1.9K
When electromagnetic radiation passes through a material, atoms or molecules transition from a lower to a higher energy state by absorbing radiation corresponding to the energy difference between the two states. The absorption of infrared (IR) radiation causes transitions between vibrational energy levels in a molecule. Therefore, IR spectroscopy is a useful analytical tool for determining the molecular structure of molecules.
Different compounds display unique properties due to their...
Different compounds display unique properties due to their...
1.9K


