Self-Supervised Video-Centralised Transformer for Video Face Clustering.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Traditional face clustering methods struggle with temporal dynamics in videos.
• Existing frame-level representations aggregated by pooling may lose crucial video information.
• There is a need for self-supervised learning methods that create clustering-friendly face representations for videos.

Purpose of the Study:

• To develop a novel video-centralised transformer for enhanced face clustering in videos.
• To propose a self-supervised framework for training video-level face representations.
• To address the unexplored area of face clustering in egocentric videos.

Main Methods:

• A video-centralised transformer architecture is employed to directly learn video-level representations.
• A self-supervised learning framework is proposed to train the transformer model.
• The first large-scale egocentric video face clustering dataset, EasyCom-Clustering, is introduced.

Main Results:

• The proposed video-centralised transformer surpasses previous state-of-the-art methods on benchmark datasets.
• The method demonstrates superior performance in capturing temporally-varying facial properties.
• Effective face clustering is achieved in both standard and egocentric video settings.

Conclusions:

• The video-centralised transformer offers a significant advancement in video face clustering.
• Self-supervised learning is effective in generating robust, clustering-friendly video face representations.
• This work pioneers face clustering research in egocentric videos and provides a valuable new dataset.