Researching the CNN Collaborative Inference Mechanism for Heterogeneous Edge Devices

Area of Science:

• Computer Science
• Artificial Intelligence
• Edge Computing

Background:

• Convolutional Neural Networks (CNNs) are crucial for intelligent sensors in edge computing.
• Limited resources and diverse architectures on edge devices hinder local CNN inference.
• Collaborative inference offers a solution by leveraging distributed resources and optimizing latency.

Purpose of the Study:

• To enable efficient collaborative execution of CNN inference tasks on heterogeneous, resource-constrained edge devices.
• To address the challenges of high computational demands and limited local processing power.

Main Methods:

• A pre-partitioning deployment strategy for CNNs, identifying critical operator layers for task division.
• Optimization of pipeline parallelism latency using data compression, queuing, and "micro-shifting" techniques.

Main Results:

• Significant acceleration of CNN inference in heterogeneous edge environments.
• A performance improvement of 71.6% compared to existing popular frameworks.

Conclusions:

• The proposed method effectively overcomes the limitations of local CNN inference on edge devices.
• Collaborative inference, enabled by strategic partitioning and optimization, is a viable approach for efficient edge AI.