GAT TransPruning: progressive channel pruning strategy combining graph attention network and transformer.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Large-scale AI models face deployment challenges on edge devices due to high resource demands.
• Model compression is essential for practical AI applications on resource-limited platforms.

Purpose of the Study:

• To propose a progressive channel pruning strategy, GAT TransPruning, for efficient large-scale model compression.
• To optimize the trade-off between model size and performance for edge AI deployment.

Main Methods:

• GAT TransPruning combines Graph Attention Networks (GAT) and Transformer attention mechanisms.
• It identifies and prunes less critical channel-to-channel relationships in neural networks.
• The strategy was validated on VGG and ResNet models using CIFAR-10 and CIFAR-100 datasets.

Main Results:

• Achieved an 89% channel pruning rate on VGG-19/CIFAR-100 with only a 6.58% accuracy drop.
• Resulting lightweight models demonstrated a 9.10x faster inference speed compared to original models.
• The proposed method showed superior performance retention under high pruning ratios compared to traditional methods.

Conclusions:

• GAT TransPruning effectively reduces model size and enhances inference speed for large AI models.
• The strategy maintains critical network connections, minimizing performance degradation.
• This approach facilitates the practical deployment of powerful AI models on edge computing platforms.