Optimization of Edge Resources for Deep Learning Application with Batch and Model Management.

Area of Science:

• Computer Science
• Artificial Intelligence
• Edge Computing

Background:

• Deep learning applications are increasingly popular, requiring efficient resource management in edge computing architectures.
• Optimizing deep learning models (e.g., pruning, binarization) and workload distribution are key challenges for edge resources.
• Effective resource utilization is crucial for deploying deep learning services at the edge.

Purpose of the Study:

• To propose a novel usage optimization method for edge computing resources, specifically targeting GPU utilization.
• To enhance the efficiency of deep learning inference applications in edge environments.
• To improve the performance of real-time video analysis using edge AI.

Main Methods:

• Developed a usage optimization method incorporating batch and model management for deep learning inference.
• Implemented a batch size management tool to dynamically adjust input batch sizes based on available resources.
• Integrated an on-the-fly model update capability for flexible deployment.
• Deployed the solution within a real-time video analysis application on a Kubernetes cluster using Docker containers.

Main Results:

• The proposed method effectively increases GPU resource utilization during deep learning inference.
• Empirical measurements demonstrated the impact of batch inference on GPU performance.
• The system successfully optimized edge resource usage for real-time video analysis.

Conclusions:

• The presented batch and model management strategy significantly enhances edge resource optimization for deep learning tasks.
• The method provides a practical solution for improving the efficiency of real-time AI applications at the edge.
• This approach contributes to the advancement of efficient deep learning deployment in edge computing environments.