Plastic waste identification based on multimodal feature selection and cross-modal Swin Transformer

Area of Science:

• Environmental Science
• Computer Vision
• Materials Science

Background:

• Municipal solid waste (MSW) management relies on effective plastic waste sorting for resource conservation and pollution prevention.
• Multimodal detection offers superior information capacity compared to single-modal methods for waste identification.
• Existing hyperspectral feature selection and multimodal identification methods do not fully exploit cross-modal information.

Purpose of the Study:

• To develop advanced methods for plastic waste identification using multimodal data.
• To improve the efficiency and accuracy of waste sorting processes.
• To address the limitations of current hyperspectral feature selection and multimodal identification techniques.

Main Methods:

• Construction of two RGB-hyperspectral image (RGB-HSI) multimodal instance segmentation datasets for plastic waste.
• Proposal of a feature band selection algorithm based on the Activation Weight function for hyperspectral data.
• Introduction of the multimodal Selective Feature Network (SFNet) and the Correlation Swin Transformer Block for cross-modal information fusion.

Main Results:

• The Activation Weight band selection algorithm effectively identifies influential hyperspectral bands.
• The Correlation SF-Swin Transformer achieved high F1-scores of 97.85% and 97.37% in plastic waste object detection.
• The proposed methods demonstrate enhanced multimodal recognition capabilities for plastic waste.

Conclusions:

• The developed multimodal approach significantly advances plastic waste identification in MSW sorting.
• The Activation Weight function and Correlation SF-Swin Transformer offer efficient and accurate solutions for waste management.
• The created datasets and models support further research in intelligent waste sorting and resource recovery.