Accelerating the tuning process for optimizing DNN operators by ROFT model.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural networks (DNNs) require intensive computation and optimization.
• Compiler optimizations offer performance comparable to manual methods but involve lengthy tuning.
• Existing optimization techniques face challenges with long tuning times.

Purpose of the Study:

• To propose a novel method for accelerating the DNN tuning process.
• To significantly reduce optimization time without compromising DNN performance.
• To introduce a cost model and search algorithm for efficient DNN optimization.

Main Methods:

• Developed ROFT (Roofline for Fast AutoTune), a Roofline-like cost model for evaluating DNN schedules.
• Implemented a flexible two-stage search algorithm based on the ROFT cost model.
• Evaluated ROFT on NVidia GPUs and Huawei Ascend NPUs.

Main Results:

• ROFT accelerates the tuning process by approximately 4X on NVidia GPUs and 10X on Huawei Ascend310 NPUs compared to existing methods.
• The method achieves significant speedups for typical deep neural networks.
• Inference time for some DNNs was improved by up to 7%.

Conclusions:

• The proposed ROFT method offers a substantial improvement in DNN tuning efficiency.
• ROFT provides a practical solution for reducing the computational overhead in DNN optimization.
• This approach enables faster deployment of optimized deep neural networks on diverse hardware.