基于语义信息的多模式图像融合和对象检测.
Yong Liu1, Xin Zhou2, Wei Zhong2
1School of Software Technology, Dalian University of Technology, Dalian 116620, China.
Entropy (Basel, Switzerland)
|May 27, 2023
概括
这项研究引入了一种用于红外和可见图像融合 (IVIF) 的新型深度学习网络,该网络可以增强目标检测. 拟议的方法有效地融合多模式图像,删除冗余信息以提高准确性.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 图像处理 图像处理
背景情况:
- 现有的红外和可见图像融合 (IVIF) 深度学习方法经常忽视传输特性,导致信息退化.
- 当前的核聚变技术可能会保留冗余或无效的信息,阻碍下游任务,如目标检测.
研究的目的:
- 开发一个端到端的深度学习网络,用于准确的红外和可见图像融合.
- 增强从两种图像模式中提取有效信息的功能,无需遗漏或冗余.
- 使用融合图像来提高后续目标检测任务的性能.
主要方法:
- 提出了一个由语义信息指导的多层结构搜索注意力融合网络.
- 引入神经架构搜索 (NAS) 以优化网络设计.
- 开发了一种新的多层次适应性注意模块 (MAAB),以保留基本特征并删除不相关的信息.
主要成果:
- 拟议的融合网络在M3FD数据集的主观和客观评估中取得了先进的表现.
- 在对象检测任务中,与FusionGAN.com相比,平均平均精度 (mAP) 提高了0.5%.
- 从红外和可见图像中有效保留特征信息,同时消除无用的数据.
结论:
- 开发的融合网络为红外和可见图像融合提供了强大的解决方案.
- 纳斯和MAAB的整合显著提高了核聚变质量和下游任务性能.
- 拟议的方法提供了一种可靠的方法来生成合图像,这对目标检测非常有益.
更多相关视频
07:13Author Spotlight: An Efficient and Robust Software for Automated Fusion of Multiple Preclinical Imaging Modalities
Published on: October 27, 2023
1.2K
08:25Combining Eye-tracking Data with an Analysis of Video Content from Free-viewing a Video of a Walk in an Urban Park Environment
Published on: May 7, 2019
9.0K
相关概念视频
01:20Parallel Processing
187
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
187
01:24Tagging and Fusion Proteins
6.8K
Proteins are involved in several cellular processes and biochemical reactions. Analyzing a specific protein of interest requires it to be isolated from the other proteins in the cell. This is achieved by overexpressing the specific gene in a suitable host to produce large quantities of the target protein. A tag or label is recombined with the gene to produce a fusion protein containing the target protein and the tag. The tags on these fusion proteins can then be used for easy detection and...
6.8K