Wavelet-CNet: Wavelet Cross Fusion and Detail Enhancement Network for RGB-Thermal Semantic Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Sensor Fusion

Background:

• Industrial sensing increasingly relies on fusing RGB and thermal infrared (TIR) data for enhanced scene understanding.
• Current RGB-T semantic segmentation methods exhibit limited inter-modal information exchange and fail to fully exploit thermal data characteristics.
• This leads to suboptimal feature complementarity and reduced segmentation accuracy.

Purpose of the Study:

• To develop an advanced network, Wavelet-CNet, for more effective RGB-T semantic segmentation.
• To improve feature fusion by leveraging wavelet transforms and cross-scale contextual information from thermal data.
• To enhance the performance of mobile devices in industrial scene understanding tasks.

Main Methods:

• Proposed Wavelet-CNet architecture for RGB-T semantic segmentation.
• Introduced a Wavelet Cross Fusion Module (WCFM) using wavelet transforms for multi-frequency feature extraction and reconstruction.
• Developed a Cross-Scale Detail Enhancement Module (CSDEM) to integrate TIR branch contextual information for improved localization.

Main Results:

• Wavelet-CNet achieved mIoU scores of 58.3% on MFNet and 85.77% on PST900 datasets.
• Ablation studies on the MFNet dataset confirmed the significant contributions of the WCFM and CSDEM modules.
• Demonstrated superior performance in exploiting thermal data characteristics for segmentation.

Conclusions:

• Wavelet-CNet effectively addresses limitations in existing RGB-T semantic segmentation networks.
• The proposed WCFM and CSDEM modules significantly enhance feature fusion and cross-modal interaction.
• The developed method shows strong potential for advancing industrial sensing and mobile scene understanding.