Generative Distribution Embeddings: Lifting autoencoders to the space of distributions for multiscale representation...

Area of Science:

• Machine Learning
• Computational Biology
• Generative Models

Background:

• Many scientific challenges require analyzing entire data distributions, not isolated data points.
• Existing models often struggle with multi-scale reasoning across complex datasets.
• There is a need for advanced frameworks capable of learning from and representing distributions.

Purpose of the Study:

• Introduce Generative Distribution Embeddings (GDEs), a novel framework to extend autoencoders to the space of data distributions.
• Enable learning of robust distributional representations using conditional generative models and distributional invariance.
• Demonstrate the efficacy of GDEs in capturing essential statistical properties and enabling meaningful latent space operations.

Main Methods:

• Developed a framework (GDEs) where encoders process sets of samples and decoders are replaced by generators matching input distributions.
• Incorporated conditional generative models with encoder networks satisfying distributional invariance.
• Utilized Wasserstein space embeddings to learn predictive sufficient statistics.

Main Results:

• GDEs learn latent representations where distances approximate Wasserstein ($W_2$) distance and interpolations recover optimal transport trajectories for Gaussian distributions.
• Systematic benchmarking on synthetic datasets shows superior performance compared to existing methods.
• Successfully applied GDEs to six diverse computational biology problems, including single-cell genomics, transcriptomics, and protein sequence analysis.

Conclusions:

• GDEs provide a powerful method for learning and representing probability distributions, outperforming existing approaches.
• The framework demonstrates significant potential for advancing computational biology by enabling analysis of large-scale, complex biological data.
• GDEs offer a versatile tool for various scientific domains requiring distributional reasoning.