RAPA-ConvNets: Modified Convolutional Networks for Accelerated Training on Architectures With Analog Arrays

Area of Science:

• Artificial Intelligence
• Computer Engineering
• Hardware Acceleration

Background:

• Analog arrays offer fast matrix-vector multiplication, crucial for deep learning.
• Traditional Convolutional Neural Networks (ConvNets) map inefficiently to analog arrays due to sequential computation needs.

Purpose of the Study:

• To propose a novel parallelized training method for ConvNets on analog arrays.
• To demonstrate the suitability of analog hardware for efficient ConvNet execution.

Main Methods:

• Replicating convolution kernel matrices across multiple analog arrays.
• Distributing computation randomly among these replicated arrays.
• Analytical and numerical experiments to validate the approach.

Main Results:

• Achieved significant acceleration factors proportional to the number of replicated kernel matrices.
• Demonstrated a self-regularizing effect leading to implicit learning of similar filters.
• Reported superior performance and increased robustness to adversarial attacks on various datasets.

Conclusions:

• The proposed parallelization strategy effectively adapts ConvNets for analog array hardware.
• Analog arrays are suitable for accelerating ConvNets, challenging prior assumptions.
• This approach enhances deep learning efficiency and model robustness.