使用CLIP进行可见红外图像融合的语言驱动交叉注意
Xue Wang1, Jiatong Wu1, Pengfei Zhang1
1Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China.
Sensors (Basel, Switzerland)
|August 28, 2025
概括
这项研究引入了一种新的文本导向红外和可见图像融合网络. 机器人在具有挑战性的环境中增强了多模式感知,改善了导航和环境理解.
科学领域:
- 计算机视觉
- 人工智能
- 机器人技术
背景情况:
- 在低光或复杂的环境中进行单模成像.
- 多模式融合 (可见和红外) 对机器人的感知至关重要.
- 增强的场景理解有助于导航,定位和环境感知.
研究的目的:
- 开发一个以文本为导向的红外和可见图像融合网络.
- 通过整合语义信息来提高多式融合效率.
- 产生高质量的融合图像,增强视觉细节和语义理解.
主要方法:
- 一个结合图像融合分支和文本导向模块的新框架.
- 跨领域的注意力机制,用于合并多式联络功能.
- 从文本描述中提取语义线索以指导融合的CLIP模型.
主要成果:
- 在LLVIP和TNO基准数据集上取得了强的定量表现.
- 在多模式感知任务中表现出强大且可扩展的性能.
- 产生高质量的色彩融合图像,丰富语义理解.
结论:
- 拟议的文本导向融合网络有效地整合了视觉和语言信息.
- 该框架为现实世界中的多式联络感知应用提供了一个有前途的解决方案.
- 增强的融合能力对于复杂环境中的自主系统至关重要.
相关概念视频
Confocal Fluorescence Microscopy
14.2K
Confocal microscopy is an advanced microscopic technique. The prime advantage of the confocal microscope over other microscopy techniques is its ability to block the out-of-focus light from the illuminated samples using pinholes. It is widely used with fluorescence optics to obtain high-resolution, sharp contrast images. Unlike optical microscopes, confocal microscopes use a focused beam of light laser to scan the entire sample surface at different z-planes. These microscopes are, therefore,...
14.2K
Light Acquisition
8.6K
In order to produce glucose, plants need to capture sufficient light energy. Many modern plants have evolved leaves specialized for light acquisition. Leaves can be only millimeters in width or tens of meters wide, depending on the environment. Due to competition for sunlight, evolution has driven the evolution of increasingly larger leaves and taller plants, to avoid shading by their neighbors with contaminant elaboration of root architecture and mechanisms to transport water and nutrients.
8.6K
Infrared (IR) Spectroscopy: Overview
2.3K
When electromagnetic radiation passes through a material, atoms or molecules transition from a lower to a higher energy state by absorbing radiation corresponding to the energy difference between the two states. The absorption of infrared (IR) radiation causes transitions between vibrational energy levels in a molecule. Therefore, IR spectroscopy is a useful analytical tool for determining the molecular structure of molecules.
Different compounds display unique properties due to their...
Different compounds display unique properties due to their...
2.3K
Vision
55.3K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
55.3K
Phase Contrast and Differential Interference Contrast Microscopy
9.4K
Phase-Contrast Microscopes
In-phase-contrast microscopes, interference between light directly passing through a cell and light refracted by cellular components is used to create high-contrast, high-resolution images without staining. It is the oldest and simplest type of microscope that creates an image by altering the wavelengths of light rays passing through the specimen. Altered wavelength paths are created using an annular stop in the condenser. The annular stop produces a hollow cone of...
In-phase-contrast microscopes, interference between light directly passing through a cell and light refracted by cellular components is used to create high-contrast, high-resolution images without staining. It is the oldest and simplest type of microscope that creates an image by altering the wavelengths of light rays passing through the specimen. Altered wavelength paths are created using an annular stop in the condenser. The annular stop produces a hollow cone of...
9.4K


