Fully Bayesian Autoencoders with Latent Sparse Gaussian Processes.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Statistics

Background:

• Autoencoders are powerful tools for representation learning but often lack probabilistic rigor.
• Existing Bayesian autoencoder models can suffer from high inference costs.
• Integrating Gaussian Processes with autoencoders has shown promise but faces scalability challenges.

Purpose of the Study:

• To introduce a fully Bayesian autoencoder model with efficient inference.
• To enable flexible prior specifications and posterior approximations.
• To enhance scalability and handle complex data structures like missing values.

Main Methods:

• A fully Bayesian autoencoder treating local latent variables and global parameters probabilistically.
• Amortized Markov Chain Monte Carlo (MCMC) using implicit stochastic networks for posterior sampling.
• Incorporation of Sparse Gaussian Process (GP) and Deep Gaussian Process (DGP) priors on the latent space.

Main Results:

• Demonstrated low inference costs with flexible priors and posterior approximations.
• Achieved improved scalability through Bayesian treatment of GP inducing points and hyperparameters.
• Showcased strong performance in dynamic representation learning and generative modeling tasks.

Conclusions:

• The proposed Bayesian autoencoder offers a robust and scalable framework for representation and generative learning.
• The model effectively handles complex priors and missing data, outperforming existing GP-autoencoder hybrids.
• This work advances probabilistic deep learning by integrating advanced Bayesian nonparametrics.