Learning Energy-Based Models in High-Dimensional Spaces with Multiscale Denoising-Score Matching.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computer Vision

Background:

• Energy-based models (EBMs) are versatile tools for tasks like sample synthesis and denoising.
• Standard maximum likelihood training for EBMs is computationally intensive due to sampling requirements.
• Denoising-score matching offers faster training but has historically lacked high-quality sample synthesis for high-dimensional data.

Purpose of the Study:

• To analyze and demonstrate the necessity of multi-noise level training for high-dimensional EBMs.
• To introduce a novel EBM trained with multiscale denoising-score matching.
• To establish a new performance baseline for EBMs in generative tasks.

Main Methods:

• Analysis of training dynamics for EBMs with varying noise levels.
• Development and implementation of a multiscale denoising-score matching technique.
• Empirical evaluation on high-dimensional datasets and image-inpainting tasks.

Main Results:

• Training with multiple noise levels is crucial for effective high-dimensional data synthesis.
• The proposed multiscale EBM achieves performance comparable to state-of-the-art generative models like GANs.
• The model demonstrates strong performance in density estimation and image inpainting.

Conclusions:

• Multiscale denoising-score matching significantly enhances EBM capabilities for high-dimensional generative tasks.
• The proposed EBM offers a competitive alternative to existing generative models.
• This work advances the application of EBMs in complex data generation and analysis.