Disentangled Representation Learning and Generation With Manifold Optimization.

Area of Science:

• Machine Learning
• Representation Learning
• Generative Models

Background:

• Disentanglement enhances interpretability in generative models like VAEs.
• Existing models often face a trade-off between disentanglement and generation quality.

Purpose of the Study:

• To present a representation learning framework that explicitly promotes disentanglement.
• To improve both disentanglement and generation quality in latent space models.

Main Methods:

• Proposed an objective function combining autoencoder and PCA reconstruction error.
• Utilized a Cayley ADAM algorithm for stochastic optimization on the Stiefel manifold.
• Employed an alternating minimization scheme with the Adam optimizer.

Main Results:

• Demonstrated that the framework promotes disentanglement by aligning latent space principal directions with data space variations.
• Showcased improved disentanglement and generation quality compared to VAE variants.
• Provided theoretical analysis and experimental validation.

Conclusions:

• The proposed framework offers a significant improvement over existing VAE variants.
• Achieves superior disentangled representation learning and generation quality.
• Offers a new approach for interpretable generative modeling.