A Post-training Quantization Method for the Design of Fixed-Point-Based FPGA/ASIC Hardware Accelerators for LSTM/GRU...

Area of Science:

• Computer Science
• Artificial Intelligence
• Hardware Acceleration

Background:

• Recurrent Neural Networks (RNNs) are crucial for NLP and speech tasks but suffer from high computational and memory demands.
• Existing hardware acceleration solutions using FPGAs and ASICs are limited by the lack of efficient compression techniques for RNNs.
• Previous research has focused on feed-forward networks, neglecting the optimization of RNNs for resource-constrained environments.

Purpose of the Study:

• To propose an effective post-training quantization methodology for RNNs, focusing on LSTM and GRU architectures.
• To provide guidelines for designing custom hardware accelerators on FPGAs/ASICs using fixed-point arithmetic for RNNs.
• To evaluate the trade-offs between memory reduction and accuracy for quantized RNN models.

Main Methods:

• Developed a novel post-training quantization strategy specifically for Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models.
• Applied the quantization method to pre-trained LSTM/GRU models on IMDb sentiment classification and Penn TreeBank language modeling datasets.
• Compared the performance and accuracy of quantized models against their original floating-point counterparts.

Main Results:

• Achieved up to 90% reduction in memory footprint for both LSTM and GRU models.
• Maintained less than 1% loss in accuracy for sentiment classification tasks.
• Observed a slight improvement in the Perplexity per word metric for language modeling tasks.

Conclusions:

• The proposed quantization methodology effectively reduces the memory footprint of RNNs, making them suitable for edge devices.
• The technique offers a viable path for designing efficient hardware accelerators (FPGA/ASIC) for LSTM/GRU-based applications.
• Demonstrated significant benefits in memory reduction with negligible impact on model accuracy and performance.