CHEF: A Framework for Deploying Heterogeneous Models on Clusters with Heterogeneous FPGAs.

Area of Science:

• Artificial Intelligence
• Computer Engineering
• Hardware Acceleration

Background:

• Deep neural networks (DNNs) are evolving from single-modality single-task (SMST) to complex multi-modality multi-task (MMMT) architectures.
• Heterogeneous hardware systems, particularly those utilizing FPGAs, are increasingly adopted to meet the demands of advanced DNNs.
• Existing methods for mapping DNNs onto heterogeneous FPGAs struggle with the complexity and scale of MMMT models.

Purpose of the Study:

• To develop an efficient system for implementing MMMT models on heterogeneous FPGA clusters.
• To address the challenges of deploying diverse accelerators and mapping complex DNNs in realistic hardware environments.
• To propose a novel framework that optimizes both hardware deployment and accelerator mapping.

Main Methods:

• Introduced CHEF, a framework comprising CHEF-A2F for accelerator-to-FPGA deployment and CHEF-M2A for DNN-to-accelerator mapping.
• CHEF-A2F employs a two-stage co-optimization approach for hardware deployment and accelerator mapping.
• CHEF-M2A is designed to handle general and practical MMMT model mapping scenarios.

Main Results:

• Demonstrated the first implementation of MMMT models on real heterogeneous FPGA clusters.
• Achieved near-optimal latency for MMMT model execution.
• Reduced search time by 10,000 times compared to exhaustive search methods.

Conclusions:

• CHEF provides an efficient and practical solution for deploying complex MMMT DNNs on heterogeneous FPGAs.
• The framework significantly advances the state-of-the-art in hardware acceleration for large-scale AI models.
• This work paves the way for more sophisticated AI applications on specialized hardware platforms.