语义指导融合网络用于交叉模式的语义细分
Pan Zhang1, Ming Chen1, Meng Gao1
1College of Information, Shanghai Ocean University, No. 999 Hucheng Ring Road, Shanghai 201306, China.
Sensors (Basel, Switzerland)
|April 27, 2024
概括
本研究介绍了语义指导融合网络 (SGFN),用于改进多式模式细分. SGFN有效地融合了各种数据类型,提高了复杂的细分任务的性能.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 多模式数据融合对于高级细分任务至关重要.
- 整合各种传感器数据,如深度和热成像,在跨模态特征提取中提出了挑战.
- 现有的方法难以有效地合并不同模式的独特特征.
研究的目的:
- 引入一个创新的跨模式融合网络,即语义指导融合网络 (SGFN).
- 加强双模特征提取并有效地整合不同的细分模式.
- 为了应对在统一的框架内合并不同的数据特征的挑战.
主要方法:
- 开发了用于多模式细分的语义指导融合网络 (SGFN).
- 引入了一个语义指导模块 (SGM) 来促进特征提取.
- 利用可学习的语义指导卷积 (SGC) 来合并来自不同模式的强度和梯度数据.
主要成果:
- 在多个基准数据集 (NYU Depth V2, SUN-RGBD, Cityscapes, MFNet, ZJU) 中,SGFN表现出卓越的性能和概括性.
- 在使用双模 SGFN 的 DELIVER 数据集上,实现了与 CMXNET 等领先模型可比的性能.
- 验证了SGM和SGC在改善跨模式融合方面的有效性.
结论:
- 该SGFN提供了一个强大的和有效的解决方案,用于多式联运细分.
- 提出的语义指导方法显著提高了各种数据模式的整合.
- 对于计算机视觉任务,SGFN代表了跨模式融合的重大进步.
相关概念视频
Visual System
579
Light enters the eye through the cornea, a transparent, dome-shaped surface covering the surface of the eyeball that helps to direct and focus incoming light. This light is then channeled toward the pupil, an adjustable opening whose size is controlled by the iris. The iris, a pigmented muscle, regulates the amount of light entering the eye by contracting or dilating the pupil, thereby ensuring optimal light levels for clear vision.
Once through the pupil, the light passes through the lens, a...
Once through the pupil, the light passes through the lens, a...
579
Tagging and Fusion Proteins
6.6K
Proteins are involved in several cellular processes and biochemical reactions. Analyzing a specific protein of interest requires it to be isolated from the other proteins in the cell. This is achieved by overexpressing the specific gene in a suitable host to produce large quantities of the target protein. A tag or label is recombined with the gene to produce a fusion protein containing the target protein and the tag. The tags on these fusion proteins can then be used for easy detection and...
6.6K
Association Areas of the Cortex
5.3K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
5.3K
Synesthesia
122
Synesthesia is a remarkable condition where stimulation of one sensory or cognitive pathway leads to automatic, involuntary experiences in a second sensory or cognitive pathway. People with synesthesia experience a blending or crossing of their senses, such as sight and sound, leading to cross-modal sensations. In this condition, the stimulation of one sense, such as hearing a number or musical note, triggers an experience of another sense, like sensing a specific color, taste, or smell. People...
122
Force Classification
1.2K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.2K
Structural Classification of Joints
3.4K
Joints, also known as articulations, are classified based on their structural characteristics, i.e., based on whether the articulating surfaces of the adjacent bones are directly connected by fibrous connective tissue or cartilage, or whether the articulating surfaces contact each other within a fluid-filled joint cavity. These differences serve to divide the joints of the body into three structural classifications.
A fibrous joint is where the adjacent bones are united by fibrous connective...
A fibrous joint is where the adjacent bones are united by fibrous connective...
3.4K


