Related Experiment Video
Updated: Jul 4, 2025
03:31
Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications
Published on: December 15, 2023
542
Multi-modal deep learning networks for RGB-D pavement waste detection and recognition
1School of Automation Science and Engineering, Faculty of Electronic and Information Engineering, Xi'an Jiaotong University, Xi'an 710049, Shaanxi, China.
Waste Management (New York, N.Y.)
|February 7, 2024
Summary
This study introduces a new multi-modal learning approach for detecting and recognizing pavement waste using RGB and depth images. The proposed MM-Net significantly improves waste management efficiency for intelligent cleaning robots.
Area of Science:
- Computer Science
- Artificial Intelligence
- Robotics
Background:
- Current pavement waste management relies on inefficient manual labor.
- Existing deep learning methods for waste detection are limited by RGB image data.
Purpose of the Study:
- To develop an efficient multi-modal learning solution for pavement waste detection and recognition.
- To address the limitations of RGB-only methods by incorporating depth information.
Main Methods:
- Construction of the high-quality outdoor pavement waste dataset (OPWaste) with color and depth images.
- Development of a novel multi-modal multi-scale network (MM-Net) incorporating multi-scale refinement (MRM) and interaction (MIM) modules.
- Exploration and comparison of six different multi-modal fusion techniques.
Main Results:
- MM-Net, utilizing an image addition fusion method, achieved state-of-the-art performance.
- The model reached 97.3% on mAP0.5 and 84.4% on AR metrics in experimental evaluations.
- Comparative experiments demonstrated the superiority of MM-Net over other deep learning models.
Conclusions:
- Multi-modal learning is crucial for advancing intelligent waste recycling systems.
- The proposed MM-Net offers a promising solution for automatic outdoor waste management using intelligent cleaning robots.
