Neuromorphic spike-based large language model.

Area of Science:

• Neuromorphic Engineering
• Computational Neuroscience
• Artificial Intelligence

Background:

• Large-language-models (LLMs) face challenges with high energy consumption and limited interpretability.
• Current LLM architectures are computationally intensive and lack biological plausibility.

Purpose of the Study:

• To propose a unified neuromorphic spike-based large-language-model (NSLLM) framework.
• To enhance both energy efficiency and interpretability of LLMs simultaneously.
• To offer a novel neuroscientific perspective on LLMs.

Main Methods:

• Transforming LLMs into NSLLMs using mathematical modeling, quantization, and sparsification.
• Converting LLM behaviors into neural dynamics like spike trains.
• Leveraging a hardware-algorithm co-design paradigm with a custom MatMul-free hardware core on an FPGA.
• Utilizing computational neuroscience tools for analyzing information encoding.

Main Results:

• Achieved dynamic power consumption of 13.849 W and inference throughput of 161.8 tokens/sec for a 1.5-billion-parameter NSLLM.
• Demonstrated significant improvements in energy efficiency (19.8%), memory usage (21.3%), and inference throughput (2.2x) compared to A800 GPU.
• Successfully eliminated matrix multiplication (MatMul) in the NSLLM hardware core.

Conclusions:

• The NSLLM framework offers a unified approach to boost LLM energy efficiency and interpretability.
• Conceptualizing LLMs as neural populations enhances understanding through computational neuroscience.
• The findings provide valuable insights for designing future neuromorphic chips for large models.