RGB-D fusion models for construction and demolition waste detection

The development of urbanization has brought a large amount of construction and demolition waste (CDW), which occupy land and cause adverse ecological effects. To effectively solve the negative impact of CDW, it needs to be recycled. Accurate waste classification is key to successful waste management. However, the current waste classification methods mainly use color images to classify, which cannot meet the needs of accurate classification. This paper built an RGB-depth (RGB-D) detection platform, using a color camera and a laser line-scanning sensor to collect RGB images and depth images. In order to use RGB images and depth images for feature fusion more effectively, this paper proposed three fusion models: RGB-D concat、RGB-D Ci-add and RGB-D Ci-concat. All these models based on an instance segmentation network called mask region convolutional neural network (Mask R-CNN), which can accurately segment the contours of each object while classifying them. The experimental results show that the mAPs of the RGB-D Ci-add / concat model are 1.33% to 1.72% higher than those of the RGB model, and the classification accuracy is 1.92% ∼ 2.27% higher. In addition, all the proposed models can meet the real-time requirement of online detection. Due to the excellent comprehensive performance of the RGB-D Ci-concat model, it can be regarded as the final detection model of the robot, which can improve the sorting efficiency of CDW further.