CoSeg: Cognitively Inspired Unsupervised Generic Event Segmentation.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Cognitive Science

Background:

• Human event segmentation is linked to event anticipation.
• Existing methods for event segmentation often rely on clustering.
• A need exists for effective self-supervised learning frameworks for generic event segmentation.

Purpose of the Study:

• To propose a novel end-to-end self-supervised learning framework for event segmentation and boundary detection.
• To leverage the principle of prediction deviation, similar to human perception, for identifying event boundaries.
• To achieve accurate segmentation of generic events.

Main Methods:

• Utilizes a transformer-based feature reconstruction scheme to detect event boundaries via reconstruction errors.
• Employs a temporal contrastive feature embedding (TCFE) module for learning semantic visual representations.
• Focuses on semantic feature-level reconstruction rather than pixel-level reconstruction.

Main Results:

• The proposed framework demonstrates superior performance in event boundary detection compared to existing methods.
• Achieved significantly better results across four publicly available datasets.
• The F1 score was used as the primary evaluation metric, with MoF and IoU also calculated.

Conclusions:

• The developed self-supervised learning framework is effective for generic event segmentation.
• The approach successfully identifies event boundaries by analyzing reconstruction errors on semantic features.
• The method offers a promising alternative to traditional clustering-based approaches in computer vision.