HDF-Net:用于红外和可见图像融合的分层双分支特征提取融合网络
Yanghang Zhu1,2, Mingsheng Huang1,2, Yaohua Zhu1,2
1University of Chinese Academy of Sciences, Beijing 100049, China.
Sensors (Basel, Switzerland)
|September 19, 2025
概括
一个新的HDF-Net模型通过分离特征来改善红外和可见图像的融合. 这个网络增强了场景的感知和理解,在关键指标上表现优于现有的方法.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 图像处理 图像处理
背景情况:
- 现有的红外和可见图像融合 (IVIF) 方法难以有效地分离模式特定和共享特征,限制了融合质量.
- 这种限制阻碍了最佳的场景感知和理解,因为无法充分利用红外和可见传感器的互补数据.
研究的目的:
- 引入一个新的分层双分支融合网络 (HDF-Net),以改善IVIF.
- 在图像融合中明确解决分离模式特定和共享特征的挑战.
主要方法:
- 高频网络将图像分解成低频 (共享结构) 和高频 (特定细节) 组件.
- 一个针轮卷积变压器 (PCT) 模块通过卷积处理和方向注意力来增强低频特征提取.
- 一个层次特征改进 (HFR) 块通过基于内核的注意力和扩展的卷积来适应地集成多尺度特征.
主要成果:
- 在四个公共IVIF数据集中,HDF-Net与12种最先进的方法相比表现出了卓越的表现.
- 在RoadScene数据集中,HDF-Net在六个指标 (EN,SD,AG,SF,SCD,SSIM) 中取得了最佳表现,比第二最佳方法有显著的百分比改进.
- 结果验证了网络在实际IVIF应用中的通用性和有效性.
结论:
- 拟议的HDF-Net有效地分离和融合来自红外和可见图像的特征,提高融合质量.
- 新型PCT和HFR模块有助于强大的特征提取和自适应多尺度集成,从而实现最先进的性能.
- 在各种实际的IVIF场景中,HDF-Net为改善场景感知和理解提供了一个有前途的解决方案.
相关概念视频
IR Frequency Region: Fingerprint Region
1.9K
IR spectra are divided into two main regions: the diagnostic region and the fingerprint region. The diagnostic region of the spectrum lies above 1500 cm−1. The absorptions resulting from single-bond vibrations of the N–H, C–H, and O–H stretch at higher wavenumbers and appear on the left side of the spectrum. The stretching absorptions of the C≡C and C≡N occur between 2100–2300 cm−1. In contrast, those arising from stretching absorptions of the...
1.9K
Infrared (IR) Spectroscopy: Overview
4.7K
When electromagnetic radiation passes through a material, atoms or molecules transition from a lower to a higher energy state by absorbing radiation corresponding to the energy difference between the two states. The absorption of infrared (IR) radiation causes transitions between vibrational energy levels in a molecule. Therefore, IR spectroscopy is a useful analytical tool for determining the molecular structure of molecules.
Different compounds display unique properties due to their...
Different compounds display unique properties due to their...
4.7K
Association Areas of the Cortex
8.9K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
8.9K
Visual System
1.7K
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
1.7K
Deconvolution
548
Deconvolution, also known as inverse filtering, is the process of extracting the impulse response from known input and output signals. This technique is vital in scenarios where the system's characteristics are unknown, and they must be inferred from the observable signals.
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
548
Vision
59.4K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
59.4K
