Horizontally Distributed Inference of Deep Neural Networks for AI-Enabled IoT.

Area of Science:

• Computer Science
• Artificial Intelligence
• Internet of Things

Background:

• The proliferation of Artificial Intelligence (AI) and the Internet of Things (IoT) drives the "smart everything" paradigm.
• Edge devices in IoT infrastructures are often resource-constrained, limiting their ability to execute complex AI models.
• There is a growing need for efficient AI model execution directly on edge devices.

Purpose of the Study:

• To provide a comprehensive overview of recent research at the intersection of AI and IoT.
• To focus on mechanisms enabling collaborative inference across edge devices for complex Deep Neural Networks (DNNs).
• To address the challenges of executing state-of-the-art DNNs on resource-constrained IoT infrastructures.

Main Methods:

• Reviewing and discussing salient approaches for collaborative inference on edge devices.
• Elaborating on partitioning schemes and parallelism paradigms for DNNs.
• Analyzing underlying workflows, communication patterns, and architectural aspects of DNNs driving these techniques.

Main Results:

• Identification of key strategies for partitioning and parallelizing DNNs across distributed edge devices.
• Schematic discussion of workflows and communication patterns in collaborative edge AI.
• Highlighting design and operational challenges and their corresponding solutions.

Conclusions:

• Collaborative inference mechanisms are crucial for deploying complex DNNs on resource-constrained IoT devices.
• Effective partitioning and parallelism strategies are key to overcoming infrastructure limitations.
• Ongoing research addresses challenges to enable in situ execution of advanced AI models in smart environments.