Do generative models learn rare generative factors?

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Generative Models

Background:

• Generative models are crucial AI tools for unsupervised learning and data variability.
• Diffusion Models (DMs), Generative Adversarial Networks (GANs), and Variational Autoencoders (VAEs) excel at generating realistic data.
• Understanding how these models handle rare generative factors is key to improving their robustness.

Purpose of the Study:

• To investigate the internalization and replication of rare generative factors by DMs, GANs, and VAEs.
• To identify the underlying reasons for memorization of these rare factors.
• To evaluate mitigation strategies for improving generative model performance.

Main Methods:

• Systematic empirical study of DMs, GANs, and VAEs.
• Analysis of how models handle infrequent data variations.
• Experimental evaluation of mitigation techniques like spectral decoupling.

Main Results:

• A significant tendency for DMs, GANs, and VAEs to memorize rare generative factors was observed.
• The study identified specific reasons contributing to this memorization behavior.
• Spectral decoupling was found to mitigate memorization to some extent.

Conclusions:

• Generative models exhibit a memorization bias for rare data factors.
• Addressing this bias is essential for enhancing the reliability of AI-generated data.
• Further research into techniques like spectral decoupling is warranted.