FPGA implementation of deep-learning recurrent neural networks with sub-millisecond real-time latency for...

Advances in neurotechnology are expected to provide access to thousands of neural channel recordings including neuronal spiking, multiunit activity and local field potentials. In addition, recent studies have shown that deep learning, in particular recurrent neural networks (RNNs), provide promising approaches for decoding of large-scale neural data. These approaches involve computationally intensive algorithms with millions of parameters. In this context, an important challenge in the application of neural decoding to next generation brain-computer interfaces for complex human tasks is the development of low-latency real-time implementations. We demonstrate a Field-Programmable Gate Array (FPGA) implementation of Long Short-Term Memory (LSTM) RNNs for decoding 10,000 channels of neural data on a mobile lowpower embedded system platform called "NeuroCoder". We provide a proof of concept in the context of decoding 20dimensional spectrotemporal representation of spoken words from simulated 10,000 neural channels. In this particular case, the LSTM model included 4,042,420 parameters. In addition to providing multiple communication interfaces for the BCI system, the NeuroCoder platform can achieve sub-millisecond real-time latencies.