Glass Segmentation With RGB-Thermal Image Pairs

Area of Science:

• Computer Vision
• Image Processing
• Machine Learning

Background:

• Accurate glass segmentation is crucial for various applications, including autonomous driving and robotics.
• Traditional methods using only RGB images struggle with transparent or reflective glass surfaces.
• Thermal imaging offers complementary information due to differing transmission properties of glass for visible and thermal light.

Purpose of the Study:

• To develop an advanced method for glass segmentation by effectively fusing RGB and thermal image data.
• To improve the distinguishability of glass regions in complex scenes.
• To introduce a novel neural network architecture for multi-modal image fusion.

Main Methods:

• A new neural network architecture integrating Convolutional Neural Networks (CNNs) and Transformers.
• A novel multi-modal fusion module utilizing attention mechanisms to combine RGB and thermal data.
• Collection and annotation of a new dataset comprising 5551 RGB-thermal image pairs.

Main Results:

• The proposed method demonstrates superior performance in glass segmentation compared to existing approaches.
• Qualitative and quantitative evaluations confirm the effectiveness of fusing RGB and thermal data.
• The attention-based fusion module successfully leverages complementary information from both modalities.

Conclusions:

• Fusing RGB and thermal images significantly enhances glass segmentation accuracy.
• The proposed neural network architecture and fusion module are effective for this task.
• The publicly available dataset and code will facilitate further research in multi-modal segmentation.