An Energy-Based Prior for Generative Saliency.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Saliency prediction models generate heatmaps indicating visual attention.
• Existing generative models often use simple Gaussian priors, limiting expressiveness.
• Accurate uncertainty estimation in saliency prediction remains a challenge.

Purpose of the Study:

• To propose a novel generative saliency prediction framework with an informative energy-based prior.
• To enable more reliable pixel-wise uncertainty estimation in saliency maps.
• To improve the expressiveness of latent space representation in generative models.

Main Methods:

• Developed a saliency generator network with a continuous latent variable space.
• Defined an energy-based model as a prior distribution on the latent space.
• Employed Markov chain Monte Carlo (MCMC) with Langevin dynamics for joint training.
• Explored adversarial and variational inference algorithms as alternative training methods.

Main Results:

• The proposed model achieves accurate saliency predictions on RGB and RGB-D datasets.
• Generates reliable pixel-wise uncertainty maps, reflecting model confidence.
• The energy-based prior enhances latent space representation compared to isotropic Gaussian priors.
• Results are consistent with human perception of visual attention.

Conclusions:

• The generative saliency model with an energy-based prior offers superior performance in both prediction accuracy and uncertainty estimation.
• This approach provides a more robust and expressive generative framework for saliency prediction.
• The findings have implications for understanding and modeling visual attention in AI systems.