Detecting floating litter in freshwater bodies with semi-supervised deep learning

Area of Science:

• Computer Vision
• Environmental Monitoring
• Machine Learning

Background:

• Supervised Deep Learning methods are widely used for quantifying floating litter but require extensive labeled data, which is scarce and costly to obtain.
• Existing methods struggle with generalization across diverse locations, environmental conditions, and sensor settings due to reliance on generic pre-trained features.
• Fine-tuning on large generic datasets like ImageNet offers improvements but is limited for specialized tasks like riverine litter detection.

Purpose of the Study:

• To develop and validate a two-stage semi-supervised learning method for detecting floating litter in rivers and canals.
• To enhance the generalization capability of litter detection models to unseen locations and conditions.
• To reduce the dependency on large amounts of labeled data for effective litter quantification.

Main Methods:

• Utilized Swapping Assignments between multiple Views (SwAV), a self-supervised learning approach, to pre-train a ResNet50 backbone on approximately 100,000 unlabeled images.
• Developed a Faster R-CNN architecture by adding new layers to the SwAV pre-trained backbone and fine-tuned it using a limited set of labeled images (around 1,800 images with 2,600 litter items).
• Validated the methodology on data from Delft (Netherlands) and Jakarta (Indonesia), testing zero-shot generalization on data from Ho Chi Minh City (Vietnam), Amsterdam, and Groningen (Netherlands).

Main Results:

• The semi-supervised method matched or surpassed the supervised learning benchmark on familiar training locations, particularly showing better performance with minimal fine-tuning data (around 200 images).
• Achieved significant improvements in generalization to unseen locations, with average precision increasing by up to 12.7%.
• Demonstrated reduced false positive predictions compared to supervised methods, especially when limited labeled data was available.

Conclusions:

• The proposed two-stage semi-supervised learning approach, leveraging SwAV pre-training, offers superior feature extraction for floating litter detection.
• This method effectively addresses the data scarcity issue in environmental monitoring tasks and enhances model generalization.
• The findings suggest a promising direction for developing foundational models for AI applications in environmental monitoring, aiding the global challenge of water pollution.