Video Object Discovery and Co-Segmentation with Extremely Weak Supervision.

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Existing video co-segmentation methods struggle with practical videos containing many irrelevant frames.
• Accurate object discovery and segmentation in videos with sparse object presence is a significant challenge.

Purpose of the Study:

• To develop a robust spatio-temporal energy minimization formulation for simultaneous video object discovery and co-segmentation.
• To address the limitations of current methods in handling videos with irrelevant frames and sparse object occurrences.

Main Methods:

• A spatio-temporal auto-context model combined with appearance modeling for superpixel labeling.
• Propagation of superpixel labels to frame level using multiple instance boosting with spatial reasoning.
• Bootstrapping with minimal frame-level labels (1-3 frames) to identify target objects.

Main Results:

• The method demonstrates efficacy in single-video object segmentation, multi-video co-segmentation, and joint discovery/co-segmentation.
• Experiments on SegTrack, a video co-segmentation dataset, MOViCS, and a new XJTU-Stevens dataset validate the approach.
• The proposed method achieves state-of-the-art performance across all tested scenarios.

Conclusions:

• The presented spatio-temporal energy minimization formulation offers an effective solution for video object discovery and co-segmentation.
• The method's ability to handle irrelevant frames and require minimal initialization makes it practical for real-world applications.
• This work advances the field by providing a superior approach to segmenting and discovering objects in challenging video datasets.