NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Current deep neural network interpretation often focuses on individual neurons, limiting holistic understanding of concept encoding.
• Interpreting complex models requires methods that capture collective neuron behavior and concept interactions.

Purpose of the Study:

• To introduce Neurocartography, an interactive system for scalable summarization and visualization of concepts learned by neural networks.
• To enable a comprehensive understanding of how multiple neurons collectively encode concepts and contribute to predictions.

Main Methods:

• Developed scalable summarization techniques: neuron clustering based on semantic similarity and neuron embedding based on concept co-occurrence.
• Implemented efficient computation of neuron pair relationships in linear time complexity.
• Integrated visualization techniques including Neuron Projection View and Graph View for concept association and cluster summarization.

Main Results:

• Neurocartography successfully scales to large datasets like ImageNet (1.2M images).
• Human evaluations confirm the discovery of neuron groups representing coherent, human-meaningful concepts.
• Demonstrated ability to reveal concept cascades and complex relationships within neural networks.

Conclusions:

• Neurocartography provides an effective, scalable solution for interpreting deep neural networks beyond the neuron level.
• The system facilitates novel discoveries and a deeper understanding of concept formation and interaction in AI models.
• The open-source visualization tool is accessible via modern web browsers.