相关实验视频

Updated: Jun 15, 2025

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

Published on: April 21, 2023

CTFusion:基于CNN变压器的自我监督学习,用于红外和可见图像融合

Keying Du^1,2, Liuyang Fang¹, Jie Chen²

¹Yunnan Key Laboratory of Digital Communications, Yunnan Communications Investment & Construction Group Company Limited, Kunming, China.

Mathematical biosciences and engineering : MBE

|August 23, 2024

概括

CTFusion是一种新的自主监督学习框架,通过有效整合互补信息来增强红外和可见图像融合 (IVIF). 这种方法消除了对配的训练数据的需求,在IVIF任务中实现了卓越的性能.

科学领域:

计算机视觉计算机视觉
人工智能的人工智能
图像处理图像处理

背景情况:

红外和可见图像融合 (IVIF) 集成了来自多模式来源的信息.
现有的融合方法通常需要广泛的配对图像数据集进行训练.
在没有地面真相数据的情况下开发高效的IVIF技术是一个重大挑战.

研究的目的:

为IVIF引入CTFusion,这是一个新的卷积神经网络 (CNN) - 变压器框架.
为了利用自我监督的学习培训IVIF模型,而不需要地面真相融合图像.
提高从红外和可见图像中提取通用特征的效果.

主要方法:

CTFusion采用了一个编码器-解码器架构,具有基于CNN-变压器的特征提取 (CTFE) 模块.
自主监督学习是通过面具重建借口任务来利用,该任务是为IVIF量身定制的.
该框架旨在模拟源图像中的本地和全球依赖关系.

主要成果:

与基于传统和深度学习的五种方法相比,CTFusion表现优越.
对三个基准数据集的评估显示,主观和客观评估都有显著的改善.
自主监督的面具重建任务能够有效地学习图像特征.

结论:

CTFusion为红外和可见图像融合提供了一种有效的自我监督的方法.
拟议的CNN-Transformer架构成功地捕获了基本的图像特征.
这种方法通过减少对配对培训数据的依赖并实现最先进的结果来推进IVIF.

关键词:

图像融合图像融合图像融合自主监督学习学习变压器的变压器是一个变压器.

更多相关视频

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Published on: December 15, 2023

Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique

Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique

Published on: July 5, 2024

相关概念视频

Infrared (IR) Spectroscopy: Overview

Infrared (IR) Spectroscopy: Overview

When electromagnetic radiation passes through a material, atoms or molecules transition from a lower to a higher energy state by absorbing radiation corresponding to the energy difference between the two states. The absorption of infrared (IR) radiation causes transitions between vibrational energy levels in a molecule. Therefore, IR spectroscopy is a useful analytical tool for determining the molecular structure of molecules.
Different compounds display unique properties due to their...

IR Frequency Region: Fingerprint Region

IR Frequency Region: Fingerprint Region

IR spectra are divided into two main regions: the diagnostic region and the fingerprint region. The diagnostic region of the spectrum lies above 1500 cm−1. The absorptions resulting from single-bond vibrations of the N–H, C–H, and O–H stretch at higher wavenumbers and appear on the left side of the spectrum. The stretching absorptions of the C≡C and C≡N occur between 2100–2300 cm−1. In contrast, those arising from stretching absorptions of the...