Deep learning-based image deconstruction method with maintained saliency.

Area of Science:

• Neuroscience
• Computer Vision
• Cognitive Science

Background:

• Visual attention is crucial for processing information, driven by both bottom-up (saliency-based) and top-down (goal-directed) mechanisms.
• Understanding the neural correlates of these distinct attentional processes requires controlled visual stimuli.

Purpose of the Study:

• To develop a method for generating pairs of natural and unnatural images with matched saliency maps.
• To investigate the neural activity underlying top-down and bottom-up visual attention using these novel stimuli.

Main Methods:

• Proposed a deep neural network-based image transformation technique to generate images with consistent saliency maps.
• Incorporated higher-dimensional latent variables for unnatural image generation with diverse local structures.
• Developed Kullback-Leibler divergence regularization to prevent latent space collapse.
• Conducted human experiments measuring eye movements and functional magnetic resonance imaging (fMRI).

Main Results:

• Successfully generated diverse unnatural images sharing saliency maps with natural counterparts.
• Observed distinct neural activity patterns associated with top-down and bottom-up attention.
• Demonstrated the utility of the generated stimuli in probing attentional mechanisms.

Conclusions:

• The novel image transformation method effectively creates stimuli for studying visual attention.
• The findings provide insights into the neural differentiation of bottom-up and top-down attentional processing.