Energy Guided Diffusion for Generating Neurally Exciting Images

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Computer vision

Background:

• Most exciting inputs (MEIs) are crucial for studying visual system tuning.
• Complex models in higher visual areas challenge traditional MEI synthesis methods like gradient ascent (GA).
• GA can overfit and reduce MEI transferability across different neural network architectures.

Approach:

• Introduced a novel attention readout for convolutional models in macaque V4, outperforming ResNet.
• Proposed Energy Guidance (EGG), a diffusion-based method to generate MEIs, addressing limitations of GA.
• Demonstrated EGG's effectiveness in generating MEIs for macaque V4 models.

Key Points:

• EGG generates MEIs that generalize better across architectures than GA for V4 models.
• EGG preserves within-architecture activation and reduces computation time by 4.7x.
• EGG can generate other stimuli, like natural images and reconstructions, with improved cross-architecture generalization.

Conclusions:

• EGG offers a flexible and generalizable framework for studying visual system coding properties.
• The method is simple to implement, requires no model retraining, and can be adapted for other analyses like invariances.
• EGG enhances the study of neuronal activity and visual system characterization using natural images.