SpVOS: Efficient Video Object Segmentation With Triple Sparse Convolution.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Semi-supervised video object segmentation (Semi-VOS) methods leverage initial annotations for subsequent frame segmentation.
• Memory-matching-based Semi-VOS pipelines utilize temporal information for high-quality results.
• Existing methods face heavy computational overhead due to dense convolutions on high-resolution feature maps.

Purpose of the Study:

• To propose a sparse baseline for video object segmentation (VOS) named SpVOS.
• To reduce the computational costs of VOS frameworks.
• To maintain high segmentation performance while decreasing computational demands.

Main Methods:

• Developed a novel triple sparse convolution with a triple gate mechanism.
• The triple gate adaptively controls sparse convolution application based on spatial and temporal redundancy.
• Employed a mixed sparse training strategy and a sparsity-constrained objective function.

Main Results:

• SpVOS achieved comparable performance to non-sparse VOS baselines on DAVIS and Youtube-VOS datasets.
• Demonstrated significant computational savings, up to 42% FLOPs reduction.
• Outperformed other state-of-the-art sparse VOS methods.

Conclusions:

• SpVOS offers an effective solution for reducing computational overhead in Semi-VOS.
• The method shows strong potential for applications in resource-constrained scenarios.
• Balancing segmentation performance and computational cost is achievable with sparse techniques.