Non-equilibrium active noise enhances generative memory in diffusion models.

Area of Science:

• Artificial Intelligence
• Thermodynamics
• Statistical Mechanics

Background:

• Generative diffusion models typically use white Gaussian noise and schedules.
• This process destroys and reconstructs information for sampling high-dimensional distributions.

Purpose of the Study:

• To investigate the impact of active, temporally correlated noise on generative diffusion models.
• To explore how non-equilibrium thermodynamics alters information processing in these models.

Main Methods:

• Driving generative processes out of equilibrium with active, non-Markovian noise.
• Utilizing Fisher information analysis to quantify information decay rates.
• Analyzing the emergence of memory effects in auxiliary degrees of freedom.

Main Results:

• Active noise creates a 'memory effect,' storing semantic information in temporal correlations.
• This mechanism significantly retards information decay compared to passive Brownian motion.
• The memory effect facilitates earlier, robust symmetry breaking and resolves multi-scale structures.

Conclusions:

• Non-equilibrium protocols inspired by active matter physics offer a distinct thermodynamic pathway.
• This approach can be advantageous for recovering high-dimensional energy landscapes using generative diffusion.
• Active noise enhances information preservation and structure recovery in generative models.