Behavior Discovery and Alignment of Articulated Object Classes from Unstructured Video

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Unstructured video collections pose challenges for content organization and retrieval.
• Supervised methods require extensive manual annotation, limiting the use of large internet video datasets.
• Articulated object classes (e.g., animals) exhibit recurring motion patterns that can be exploited.

Purpose of the Study:

• To develop an automatic system for organizing unstructured video collections of articulated objects.
• To identify characteristic behaviors and achieve pixel-to-pixel alignment across different video instances.
• To enable learning of object class appearance and behaviors from unannotated internet videos.

Main Methods:

• Behavior discovery using a novel motion representation based on trajectory displacements.
• Clustering of discovered behaviors into temporal video intervals.
• Pixel-to-pixel alignment using a time-varying thin plate spline deformation model.
• Evaluation on a new, fully annotated dataset.

Main Results:

• The system successfully identifies characteristic behaviors and aligns video instances.
• Outperformed state-of-the-art methods in behavior discovery (improved dense trajectory features).
• Outperformed SIFT Flow in spatial alignment accuracy, handling appearance variations.

Conclusions:

• The proposed system offers an effective method for organizing and retrieving content from video collections.
• It provides a platform for unsupervised learning of articulated object classes from large-scale internet data.
• Automated behavior discovery and alignment overcome limitations of traditional supervised approaches.