Unsupervised Representation Learning From Sparse Transformation Analysis

Area of Science:

• Machine Learning
• Artificial Intelligence
• Representation Learning

Background:

• Representation learning literature explores principles like coding efficiency, statistical independence, causality, controllability, and symmetry.
• Existing methods often focus on specific principles for learning data representations.

Purpose of the Study:

• To propose a new method for learning representations from sequence data.
• To factorize transformations of latent variables into sparse components using probability flow models.
• To achieve disentangled and approximately equivariant representations.

Main Methods:

• Encode input data as distributions of latent activations.
• Transform latent activations using a probability flow model decomposed into rotational and potential flow fields.
• Apply a sparsity prior to encourage few active fields and infer flow speed.
• Train the model unsupervisedly using a variational objective.

Main Results:

• The model learns disentangled representations combining independent factors and transformation primitives.
• Learned flow fields represent independent transformation primitives.
• The approach achieves state-of-the-art data likelihood.
• Demonstrates state-of-the-art unsupervised approximate equivariance errors on sequence transformation datasets.

Conclusions:

• The proposed method effectively learns disentangled and approximately equivariant representations from sequence data.
• Factorizing transformations into sparse probability flow components is a promising direction for representation learning.
• The unsupervised approach offers a new way to discover underlying data symmetries and transformations.