An analog-AI chip for energy-efficient speech recognition and transcription

Area of Science:

• Artificial Intelligence
• Computer Engineering
• Materials Science

Background:

• Large artificial intelligence (AI) models demand significant energy on conventional processors.
• Analog in-memory computing (analog-AI) offers potential energy efficiency gains for AI.
• Existing analog-AI approaches struggle with accuracy and inter-tile communication for complex models.

Purpose of the Study:

• To develop an analog-AI chip with high energy efficiency and software-equivalent accuracy.
• To address the challenges of scaling analog-AI for large neural networks.
• To demonstrate a novel architecture for parallel processing and communication in analog-AI.

Main Methods:

• Designed and fabricated an analog-AI chip integrating 35 million phase-change memory devices across 34 tiles.
• Implemented massively parallel inter-tile communication and low-power analog peripheral circuitry.
• Mapped neural network models, including keyword-spotting and recurrent neural-network transducer (RNNT), onto the chip's memory devices.

Main Results:

• Achieved chip-sustained performance of up to 12.4 tera-operations per second per watt (TOPS/W).
• Demonstrated fully end-to-end software-equivalent (SWeq) accuracy for a keyword-spotting network.
• Attained near-SWeq accuracy on the MLPerf recurrent neural-network transducer (RNNT) benchmark using over 140 million memory devices across five chips.

Conclusions:

• The developed analog-AI chip offers a viable solution for energy-efficient AI processing.
• The architecture successfully demonstrates scalability and high accuracy for complex AI models.
• This work advances the field of analog computing for next-generation AI hardware.