Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Neural network architecture design relies heavily on computationally expensive trial-and-error.
• Understanding the vast search space of neural network architectures is challenging.
• Identifying effective design principles for neural networks is crucial for advancing AI.

Purpose of the Study:

• To introduce ArchExplorer, a novel visual analysis method for exploring neural architecture spaces.
• To enable a better understanding of relationships between different neural network architectures.
• To facilitate the summarization of design principles and selection of high-performing architectures.

Main Methods:

• Developed a method to calculate structural distances between neural network architectures.
• Formulated pairwise distance calculation as an all-pairs shortest path problem, optimized via single-source shortest path problems.
• Implemented hierarchical clustering based on architecture distances.
• Utilized circle-packing for visualizing architecture relationships and clusters.

Main Results:

• Achieved significant efficiency improvement in distance calculation with a time complexity reduction from O(kn^2N) to O(knN).
• Demonstrated the ability to hierarchically cluster architectures based on structural similarity.
• Created an effective visualization that reveals both global and local architecture relationships.
• Showcased ArchExplorer's utility in summarizing design principles through case studies.

Conclusions:

• ArchExplorer provides an efficient and explainable approach to navigating neural architecture spaces.
• The method aids researchers in identifying and understanding key design principles.
• ArchExplorer facilitates the selection of better-performing neural network architectures, accelerating AI development.