Learning Event Representations for Temporal Segmentation of Image Sequences by Dynamic Graph Embedding.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Self-supervised learning effectively learns event representations for temporal segmentation in image sequences.
• Current self-supervised methods are data-hungry and face domain adaptation challenges despite eliminating manual annotations.

Purpose of the Study:

• Propose Dynamic Graph Embedding (DGE), a novel approach for learning event representations.
• Address limitations of existing self-supervised methods by reducing data dependency and improving domain adaptation.

Main Methods:

• Represent image sequences as graphs encoding semantic and temporal similarity.
• Jointly learn the graph structure and its embedding through an iterative two-step process.
• Update graph based on current data representation and update representation based on graph structure.

Main Results:

• DGE learns low-dimensional embeddings reflecting temporal and semantic similarity without a training set.
• Achieved robust temporal segmentation on EDUBSeg and EDUBSeg-Desc datasets, outperforming state-of-the-art methods.
• Demonstrated generalization capabilities on Human Motion Segmentation benchmark datasets.

Conclusions:

• DGE provides an effective and data-efficient method for learning event representations for temporal segmentation.
• The proposed approach overcomes the data-hungry nature and domain adaptation issues of current self-supervised techniques.
• DGE shows strong performance and generalization for analyzing image sequences.