MUSeg: A multimodal semantic segmentation dataset for complex underground mine scenes

Visual perception is one of the core technologies for achieving unmanned and intelligent mining in underground mines. However, the harsh environment unique to underground mines poses significant challenges to visible light-based visual perception methods. Multimodal fusion semantic segmentation offers a promising solution, but the lack of dedicated multimodal datasets for underground mines severely limits its application in this field. This work develops a multimodal semantic segmentation benchmark dataset for complex underground mine scenes (MUSeg) to address this issue. The dataset comprises 3,171 aligned RGB and depth image pairs collected from six typical mines across different regions of China. According to the requirements of mine perception tasks, we manually annotated 15 categories of semantic objects, with all labels verified by mining experts. The dataset has also been evaluated using classical multimodal semantic segmentation algorithms. The MUSeg dataset not only fills the gap in this field but also provides a critical foundation for research and application of multimodal perception algorithms in mining, contributing significantly to the advancement of intelligent mining.