Network Bending: Expressive Manipulation of Generative Models in Multiple Domains

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Deep generative models create complex data like images and audio.
• Interacting with and controlling these models during generation is challenging.

Purpose of the Study:

• Introduce the network bending framework for manipulating deep generative models.
• Develop methods for analyzing and clustering features within these models.

Main Methods:

• Implemented deterministic transformations as layers in the computational graph.
• Developed an unsupervised algorithm for clustering features based on spatial activation maps.

Main Results:

• Demonstrated effective manipulation of semantically meaningful features in image and audio generation.
• Showcased the framework's ability to produce a broad range of expressive outcomes.
• Enabled unsupervised grouping of features based on spatial similarity.

Conclusions:

• Network bending offers a novel approach to control and interact with deep generative models.
• The framework facilitates direct manipulation of semantic aspects of the generative process.
• Results show broad applicability across different data domains.