SLAPP: Subgraph-level attention-based performance prediction for deep learning models.

Area of Science:

• Computer Science
• Artificial Intelligence

Background:

• The complexity of Deep Learning (DL) models necessitates accurate performance prediction for optimal design and selection.
• Existing performance prediction methods have limitations: operator-level methods neglect graph features, while graph-level methods ignore operator specifics.

Purpose of the Study:

• To develop a novel subgraph-level performance prediction method that addresses the limitations of existing operator-level and graph-level approaches.
• To improve the accuracy of predicting performance for individual operators and entire Deep Learning networks.

Main Methods:

• Propose SLAPP, a subgraph-level performance prediction method utilizing a novel Edge Aware Graph Attention Network (EAGAT).
• EAGAT effectively encodes both node and edge features for comprehensive model representation.
• Implement a mixed loss design with dynamic weight adjustment to balance operator and network performance prediction.

Main Results:

• SLAPP demonstrates superior prediction accuracy compared to traditional methods.
• The method effectively handles unseen Deep Learning models.
• Experimental evaluations show consistently better predictive performance across multiple DL models.

Conclusions:

• SLAPP offers a robust solution for Deep Learning performance prediction by integrating subgraph-level analysis.
• The proposed EAGAT and mixed loss design contribute to enhanced accuracy for both operator and network-level predictions.
• SLAPP advances the field by providing more precise and reliable performance estimations for complex DL systems.