TokenCut: Segmenting Objects in Images and Videos With Self-Supervised Transformer and Normalized Cut.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Object detection and segmentation are crucial tasks in computer vision.
• Existing methods often require large amounts of labeled data.
• Unsupervised approaches are gaining traction for their efficiency.

Purpose of the Study:

• To develop a novel graph-based algorithm for unsupervised salient object detection and segmentation.
• To leverage features from self-supervised transformers for improved performance.
• To achieve state-of-the-art results on benchmark datasets.

Main Methods:

• Organizing image/video patches into a fully connected graph.
• Labeling graph edges with similarity scores derived from transformer features.
• Formulating detection and segmentation as a graph-cut problem solved by Normalized Cut.
• Utilizing self-supervised learning for feature extraction.

Main Results:

• Achieved state-of-the-art performance on multiple image and video segmentation tasks.
• Outperformed competing methods in unsupervised object discovery by up to 6.1% on VOC07 dataset.
• Improved Intersection over Union (IoU) scores by 4.4-5.6% in unsupervised saliency detection on ECSSD, DUTS, and DUT-OMRON datasets.
• Demonstrated competitive results in unsupervised video object segmentation on DAVIS, SegTV2, and FBMS datasets.

Conclusions:

• The proposed graph-based algorithm effectively detects and segments salient objects using self-supervised features.
• The approach offers a simple yet powerful solution for unsupervised visual tasks.
• This method sets a new benchmark for unsupervised object discovery and saliency detection.