FPGA Implementation of Keyword Spotting System Using Depthwise Separable Binarized and Ternarized Neural Networks

Area of Science:

• Embedded Systems Engineering
• Artificial Intelligence
• Hardware Acceleration

Background:

• Keyword spotting (KWS) systems are crucial for human-machine interaction, often requiring separate wake-up-word (WUW) recognition and voice command classification.
• Deep learning models for KWS pose challenges for embedded systems due to computational complexity and the need for application-specific optimizations.

Purpose of the Study:

• To propose a unified hardware accelerator, the depthwise separable binarized/ternarized neural network (DS-BTNN), for simultaneous WUW recognition and command classification.
• To achieve significant area efficiency in embedded KWS systems through optimized neural network computation.

Main Methods:

• Developed a depthwise separable binarized/ternarized neural network (DS-BTNN) hardware accelerator.
• Utilized redundant bitwise operators for efficient computation of binarized neural networks (BNNs) and ternary neural networks (TNNs).
• Implemented the KWS system on a Xilinx UltraScale+ ZCU104 FPGA, processing real-time audio data.

Main Results:

• The DS-BTNN accelerator achieved a 49.3% area reduction compared to integrating separate BNN and TNN modules, with a final area of 0.558 mm2 in a 40 nm CMOS process.
• The system demonstrated high accuracy: 97.1% for BNN-based WUW recognition and 90.5% for TNN-based command classification at 170 MHz.
• The accelerator efficiently handles both WUW and command classification tasks on a single device.

Conclusions:

• The proposed DS-BTNN hardware accelerator offers a highly area-efficient solution for integrated KWS systems.
• This unified approach significantly reduces hardware complexity and improves resource utilization for embedded voice control applications.
• The demonstrated performance validates the effectiveness of binarized and ternarized neural networks in resource-constrained KWS environments.