Rapid Salient Object Detection With Difference Convolutional Neural Networks.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Salient Object Detection (SOD) models are computationally expensive, hindering deployment on resource-constrained devices.
• Existing deep neural networks for SOD lack efficiency for real-time applications.

Purpose of the Study:

• To propose efficient network designs for real-time salient object detection on resource-constrained devices.
• To combine classical SOD principles with modern Convolutional Neural Network (CNN) capabilities for improved performance.

Main Methods:

• Introduced Pixel Difference Convolutions (PDCs) to encode feature contrasts within a CNN architecture.
• Developed a difference convolution reparameterization (DCR) strategy to embed PDCs into standard convolutions, reducing computational cost.
• Proposed SpatioTemporal Difference Convolution (STDC) for efficient video SOD by enhancing 3D convolutions with spatiotemporal contrast capture.

Main Results:

• Achieved significant improvements in the efficiency-accuracy trade-off for both image and video SOD.
• On a Jetson Orin device, models with less than 1 million parameters operated at 46 FPS (images) and 150 FPS (videos).
• Outperformed existing lightweight models by over 2x (images) and 3x (videos) in speed, while maintaining superior accuracy.

Conclusions:

• The proposed SDNet and STDNet models offer a viable solution for real-time SOD on edge devices.
• The novel PDC and STDC methods effectively extract salient features while maintaining computational efficiency.
• These advancements pave the way for deploying advanced computer vision tasks on devices with limited computational power.