一种跨模态注意力驱动的多传感器融合方法,用于点云的语义细分.
Huisheng Shi1, Xin Wang2, Jianghong Zhao3,4
1Department of Remote Sensing Engineering, Henan College of Surveying and Mapping, Zhengzhou 451464, China.
Sensors (Basel, Switzerland)
|April 26, 2025
概括
交叉模式融合 (CMF) 框架有效地整合了用于自动驾驶的摄像头和LiDAR数据,在复杂场景中显著提高了语义细分的准确性和稳定性.
科学领域:
- 计算机视觉 计算机视觉
- 自主驾驶系统 自主驾驶系统
- 传感器融合式传感器
背景情况:
- 弥合摄像头图像和LiDAR点云之间的模式差距对于自动驾驶至关重要.
- 目前的融合方法在有效的跨模式特征集成方面扎.
- 语义细分性能受到无法充分利用多传感器数据的限制.
研究的目的:
- 为增强多传感器数据集成提出一种新的交叉模式融合 (CMF) 框架.
- 为了实现自动驾驶的语义细分任务的最先进性能.
- 解决现有的融合方法在处理跨模式特征方面的局限性.
主要方法:
- 将LiDAR点云投射到相机坐标上,使用透视投射获得空间深度信息.
- 采用双流特征提取网络进行单独的模式处理.
- 实施残留聚变模块 (RCF) 以跨模式的重视为多层聚变.
- 设计一个感知对齐损失,整合交叉和特征匹配术语.
主要成果:
- 在SemanticKITTI上获得了64.2%和nuScenes上79.3%的最先进的平均交叉与工会 (mIoU) 分数.
- 与现有方法相比,在复杂的驾驶场景中表现出卓越的准确性和增强的稳定性.
- 废除研究证实了跨模式注意力和感知引导的交叉损失 (Pgce) 在改善细分方面的有效性.
结论:
- 该CMF框架成功地弥合了摄像头和LiDAR数据之间的模式差距.
- 提出的注意力驱动架构和感知对齐损失显著提高语义细分性能.
- 在自动驾驶系统中,CMF为多传感器融合提供了强大而准确的解决方案.
相关概念视频
01:20Parallel Processing
125
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
125
01:21Association Areas of the Cortex
4.5K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
4.5K