Feature-Aware Uniform Tessellations on Video Manifold for Content-Sensitive Supervoxels

Area of Science:

• Computer Vision
• Video Processing
• Image Segmentation

Background:

• Supervoxels simplify video complexity for computer vision tasks.
• Content-sensitive supervoxels (CSSs) vary in size based on content density.
• Existing CSS methods may not optimally align with video features.

Purpose of the Study:

• To introduce Feature-Aware Content-Sensitive Supervoxels (FCSSs) for improved video segmentation.
• To develop a method that generates regularly shaped 3D primitive volumes aligned with video boundaries.
• To enhance the efficiency and accuracy of supervoxel computation in videos.

Main Methods:

• Mapping videos to a 3D manifold in combined color and spatiotemporal space.
• Utilizing a restricted centroidal Voronoi tessellation to balance cell uniformity and boundary alignment.
• Developing a streaming extension (streaming FCSS) for processing large videos.

Main Results:

• FCSSs are well-aligned with local object, region, and motion boundaries.
• The proposed method achieves an optimal competitive ratio of O(1).
• FCSS and streaming FCSS demonstrate state-of-the-art performance across multiple datasets and applications.

Conclusions:

• FCSS offers a superior approach to video over-segmentation compared to existing methods.
• The method effectively reduces video complexity while preserving critical boundary information.
• FCSS provides a robust and efficient solution for advanced computer vision tasks involving video data.