Video segmentation based on motion coherence of particles in a video sequence

Area of Science:

• Computer Vision
• Image Processing
• Video Analysis

Background:

• Object-oriented video segmentation is crucial for analyzing video content.
• Existing methods often lack robustness and generality for 2-D video segmentation.
• Motion coherence is a key factor in distinguishing objects within video sequences.

Purpose of the Study:

• To develop a robust and generic object-oriented video segmentation approach.
• To leverage motion coherence for improved video segmentation.
• To demonstrate the applicability of the method in video coding and other computer vision tasks.

Main Methods:

• Utilized a tracking process with adaptively sampled points (particles).
• Employed ensemble clustering to identify 2-D motion patterns from particle trajectories.
• Mapped segmentation results to a spatio-temporal feature space to identify coherent motion volumes.

Main Results:

• Achieved pixel-wise segmentation in both spatial and temporal domains.
• Demonstrated explicit temporal redundancy through spatio-temporal volumes, suggesting video coding gains.
• Presented experimental results comparing Peak Signal-to-Noise Ratio (PSNR) and prediction error entropies with existing methods.

Conclusions:

• The proposed motion coherence-based video segmentation is robust and generic.
• The method shows potential for significant gains in video coding applications.
• The approach is also applicable to video information retrieval and video understanding tasks.