Related Experiment Video
Updated: Jun 23, 2025
11:03
An Analog Macroscopic Technique for Studying Molecular Hydrodynamic Processes in Dense Gases and Liquids
Published on: December 4, 2017
8.5K
Sampling with flows, diffusion, and autoregressive neural networks from a spin-glass perspective.
Davide Ghio1, Yatin Dandi1,2, Florent Krzakala1
1Information, Learning and Physics Laboratory, École Polytechnique Fédérale de Lausanne, Lausanne CH-1015, Switzerland.
Summary
Generative models like flows and diffusion networks struggle with sampling efficiency due to phase transitions. Traditional methods like Monte Carlo and Langevin dynamics sometimes outperform these advanced techniques.
Area of Science:
- Machine Learning
- Statistical Physics
- Computational Science
Background:
- Powerful generative models (flows, diffusion, autoregressive networks) excel at data generation.
- Theoretical analysis of their performance and limitations remains a challenge.
Purpose of the Study:
- Analyze sampling efficiency of generative models on problems with known distributions.
- Compare their performance against traditional methods (Monte Carlo Markov chain, Langevin dynamics).
Main Methods:
- Focus on probability distributions from disordered systems (spin glasses, inference, constraint satisfaction).
- Map generative sampling to Bayes optimal denoising of a modified probability measure.
- Analyze sampling performance on specific problem classes.
Main Results:
- Generative models face sampling difficulties due to first-order phase transitions in the denoising path.
- Identified parameter regions where generative models fail but traditional methods succeed.
- Identified parameter regions where traditional methods fail but generative models succeed.
Conclusions:
- Generative models exhibit limitations in sampling efficiency under certain conditions.
- Traditional sampling methods offer advantages in specific scenarios.
- Hybrid approaches or careful selection of methods based on problem characteristics are warranted.
