A robust variational autoencoder using beta divergence.

Area of Science:

• Machine Learning
• Deep Learning
• Computer Vision

Background:

• Outliers in training data can significantly degrade deep learning model performance and lead to incorrect conclusions.
• Variational Autoencoders (VAEs), while popular for anomaly detection, struggle when training data contains anomalies similar to those in test data.
• Existing VAEs are sensitive to outliers, limiting their reliability in real-world anomaly detection scenarios.

Purpose of the Study:

• To develop a robust Variational Autoencoder (VAE) model capable of effectively handling outliers in training data.
• To enhance the performance and reliability of VAEs for anomaly detection tasks, particularly when training datasets include anomalies.
• To introduce a novel approach for robust anomaly detection using deep generative models.

Main Methods:

• Proposed a robust VAE (RVAE) model based on beta-divergence instead of the standard Kullback-Leibler (KL) divergence.
• Developed a new variational lower bound for VAEs that incorporates concepts from robust statistics.
• Formulated mathematical models for RVAEs applicable to Bernoulli, Gaussian, and categorical variables.

Main Results:

• The RVAE model demonstrates improved robustness to outliers in both image and categorical datasets, validated both qualitatively and quantitatively.
• The proposed RVAE maintains the same computational complexity as standard VAEs.
• A method for unsupervised hyperparameter tuning of the RVAE was successfully developed.

Conclusions:

• The developed robust variational autoencoder (RVAE) offers a computationally efficient and effective solution for anomaly detection in the presence of training data outliers.
• RVAE, utilizing beta-divergence, significantly enhances the reliability of deep generative models for anomaly detection tasks.
• The RVAE model shows promise for practical applications, such as detecting lesions in brain images, in an unsupervised manner.