ASight: Fine-Tuning Auto-Scheduling Optimizations for Model Deployment via Visual Analytics.

Area of Science:

• Computer Science
• Artificial Intelligence
• Software Engineering

Background:

• Deep learning model deployment requires hardware optimization to reduce inference latency.
• Auto-scheduling automates optimization but generates complex low-level code, hindering comprehension and manual refinement.
• Existing methods lack effective tools for understanding auto-scheduling's impact on performance.

Purpose of the Study:

• Introduce ASight, a visual analytics system to aid engineers in understanding deep learning model optimizations.
• Facilitate identification of performance bottlenecks and comprehension of auto-generated code.
• Provide insights into auto-scheduling optimization strategies.

Main Methods:

• Developed a subgraph matching algorithm for identifying graph isomorphism in Intermediate Representations.
• Tracked performance bottlenecks from low-level metrics to high-level computational graphs.
• Proposed an enhanced visualization for exploring the large search space of auto-scheduling and deriving optimization principles.

Main Results:

• ASight effectively assists engineers in identifying performance bottlenecks in deep learning models.
• The system enables comprehension of low-level code generated by auto-scheduling.
• Case studies on local machines and data centers validated ASight's effectiveness.

Conclusions:

• ASight enhances the interpretability of auto-scheduling for deep learning model optimization.
• The system supports engineers in refining hardware deployment and future manual optimizations.
• Visual analytics are crucial for managing complexity in automated AI system optimization.