Sequential Variational Autoencoder with Adversarial Classifier for Video Disentanglement.

Area of Science:

• Computer Vision
• Machine Learning
• Representation Learning

Background:

• Video disentanglement aims to separate static and dynamic features from video data.
• Existing sequential variational autoencoders with two-stream architectures struggle with feature separation, as static features often contain dynamic information.
• Dynamic features in current models lack discriminative power in the latent space.

Purpose of the Study:

• To propose an improved sequential variational autoencoder for effective video disentanglement.
• To address the limitations of two-stream architectures in separating static and dynamic video features.
• To enhance the discriminative ability of dynamic features within the latent representation.

Main Methods:

• A novel sequential variational autoencoder architecture is proposed.
• A two-stream network design is employed to capture distinct feature types.
• An adversarial classifier with supervised learning is integrated to refine feature separation and enhance discriminability.

Main Results:

• The proposed method effectively separates static and dynamic features in videos.
• The integration of an adversarial classifier significantly improves the discriminative power of dynamic features.
• Qualitative and quantitative evaluations on the Sprites and MUG datasets demonstrate superior performance compared to existing methods.

Conclusions:

• The enhanced sequential variational autoencoder with adversarial classification provides a robust solution for video disentanglement.
• The method successfully addresses the challenges of feature contamination and weak discriminability in latent representations.
• This approach offers a significant advancement in learning disentangled representations from video data.