Iterative Knowledge Exchange Between Deep Learning and Space-Time Spectral Clustering for Unsupervised Segmentation...

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Object segmentation in videos is crucial for various applications.
• Unsupervised methods are highly desirable to reduce manual annotation efforts.
• Existing methods often struggle with complex motion and appearance variations in videos.

Purpose of the Study:

• To develop a novel dual system for unsupervised object segmentation in videos.
• To leverage complementary strengths of graph-based and deep learning approaches.
• To achieve state-of-the-art performance in both unsupervised and supervised video segmentation.

Main Methods:

• A dual system integrating a space-time graph for object discovery and a deep network for feature learning.
• Iterative knowledge exchange between the graph and network modules.
• Novel spectral space-time clustering for unsupervised mask generation (pseudo-labels).
• A power iteration algorithm for efficient space-time cluster discovery on the graph.

Main Results:

• The proposed system achieves state-of-the-art performance on four challenging datasets (DAVIS, SegTrack, YouTube-Objects, DAVSOD).
• Demonstrated effectiveness of the cyclical knowledge exchange policy for improving segmentation accuracy.
• Validated theoretical claims through thorough experimental analysis.
• Achieved competitive results in both unsupervised and supervised segmentation scenarios.

Conclusions:

• The dual system effectively combines graph-based and deep learning methods for robust video object segmentation.
• Iterative knowledge exchange significantly enhances segmentation quality.
• The approach offers a powerful solution for unsupervised object segmentation in videos, with potential for supervised applications.