The Statistical Thermodynamics of Generative Diffusion Models: Phase Transitions, Symmetry Breaking, and Critical...

Area of Science:

• Machine Learning
• Statistical Physics
• Generative Modeling

Background:

• Generative diffusion models demonstrate remarkable performance in machine learning.
• Their foundations lie in non-equilibrium physics, variational inference, and stochastic calculus.

Purpose of the Study:

• To reframe generative diffusion models using equilibrium statistical mechanics.
• To analyze the phase transitions and critical phenomena within these models.

Main Methods:

• Application of equilibrium statistical mechanics tools to diffusion models.
• Analysis of generative dynamics through the lens of phase transitions and critical exponents.

Main Results:

• Generative diffusion models exhibit second-order phase transitions linked to symmetry breaking.
• These transitions are consistently in a mean-field universality class due to self-consistency in dynamics.
• Critical instability from these phase transitions is key to generative capabilities, described by mean-field critical exponents.

Conclusions:

• Equilibrium statistical mechanics provides a powerful framework for understanding generative diffusion models.
• Phase transitions and critical phenomena are central to the generative power of these models.
• The generative process can be viewed as a free energy-minimizing stochastic adiabatic transformation.