Arch-Net: Model conversion and quantization for architecture agnostic model deployment.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Deep Neural Networks (DNNs) face computational challenges hindering practical use.
• Application-Specific Integrated Circuit (ASIC) chips accelerate neural networks but often lack support for newer architectures.
• Existing ASIC hardware may not efficiently support operations like Layer Normalization or large convolutions.

Purpose of the Study:

• To develop a neural network framework (Arch-Net) compatible with existing ASIC hardware.
• To propose a methodology (Arch-Distillation) for converting diverse network architectures into Arch-Net.
• To improve the efficiency and compatibility of neural network deployment on hardware accelerators.

Main Methods:

• Introduced Arch-Net, a framework using only 3x3 Convolution, 2x2 Max-pooling, Batch Normalization, Fully Connected layers, and Concatenation.
• Developed Arch-Distillation with Residual Feature Adaptation and Teacher Attention Mechanism for network conversion.
• Implemented efficient model quantization, including sub-8-bit quantization.

Main Results:

• Arch-Net achieved performance comparable to complex architectures on image classification and machine translation tasks.
• The framework demonstrated robust performance even under sub-8-bit quantization.
• Eliminated unconventional network constructs, enhancing deployment efficiency and ASIC compatibility.

Conclusions:

• Arch-Net offers a solution for deploying neural networks on diverse ASIC chips by utilizing common, hardware-supported operators.
• Arch-Distillation effectively converts existing architectures, preserving performance while enabling quantization.
• This approach provides a new direction for structure-agnostic neural network deployment on specialized hardware.