DPACFuse:用于红外和可见图像融合的双分支渐进式学习,与补充的自我注意和卷积相结合
Huayi Zhu1, Heshan Wu1, Xiaolong Wang1
1School of Computer and Information Security, Guilin University of Electronic Science and Technology, Guilin 541004, China.
Sensors (Basel, Switzerland)
|August 26, 2023
概括
本研究介绍了DPACFuse用于红外和可见图像融合,增强跨模式功能学习. 新方法显著提高了融合图像质量和下游任务性能.
科学领域:
- 计算机视觉 计算机视觉
- 图像处理 图像处理
- 人工智能的人工智能
背景情况:
- 红外和可见图像的融合对于在单一图像中详细和突出的物体表示至关重要.
- 现有的方法往往忽视跨模式特征相互作用,限制了聚变性能.
- 挑战包括提取丰富的纹理细节,同时保留下游应用的突出对象.
研究的目的:
- 提出一种新的双分支渐进式学习,用于红外和可见图像的融合,并配合一个互补的自我注意和卷积 (DPACFuse) 网络.
- 增强信息交互和在红外和可见模式的共同特征提取.
- 为了提高融合图像的质量及其在下游任务中的实用性,例如对象检测和语义细分.
主要方法:
- 开发了跨模式特征提取 (CMEF) 模块,以加强跨模式信息交换.
- 引入了高频梯度卷积,以捕捉细粒度的细节并减轻信息丢失.
- 建议ACmix,整合自我注意力和卷积,以平衡本地/全球特征提取与降低计算成本.
主要成果:
- DPACFuse产生了富有纹理的融合图像,并有效地突出突出物体.
- 在MI,Qabf,SF和AG指标中,与最先进的方法相比,实现了~3%的改进.
- 与单独使用源图像相比,在对象检测和语义细分方面表现出~10%的增强.
结论:
- DPACFuse有效地解决了图像融合跨模式特征建模的局限性.
- 拟议的网络在图像质量和下游任务准确性方面都实现了卓越的性能.
- 这种方法在红外和可见图像融合技术方面取得了重大进展.
相关概念视频
Parallel Processing
179
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
179
Light Acquisition
8.5K
In order to produce glucose, plants need to capture sufficient light energy. Many modern plants have evolved leaves specialized for light acquisition. Leaves can be only millimeters in width or tens of meters wide, depending on the environment. Due to competition for sunlight, evolution has driven the evolution of increasingly larger leaves and taller plants, to avoid shading by their neighbors with contaminant elaboration of root architecture and mechanisms to transport water and nutrients.
8.5K
Vision
53.6K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
53.6K
Association Areas of the Cortex
5.5K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
5.5K
Infrared (IR) Spectroscopy: Overview
1.9K
When electromagnetic radiation passes through a material, atoms or molecules transition from a lower to a higher energy state by absorbing radiation corresponding to the energy difference between the two states. The absorption of infrared (IR) radiation causes transitions between vibrational energy levels in a molecule. Therefore, IR spectroscopy is a useful analytical tool for determining the molecular structure of molecules.
Different compounds display unique properties due to their...
Different compounds display unique properties due to their...
1.9K
Visual System
616
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
616


