Temporally-Varying Stimulations for Cortical Visual Neuroprosthetics using Spiking Neural Networks

Visual neuroprosthesis can help to restore a rudimentary form of sight in visually impaired subjects via electrical stimulation. These systems receive camera input images processed by a artificial neural network (ANN) to output stimulation patterns for driving a large set of electrodes. Previous optimization approaches for visual neuroprosthesis stimulation patterns provided a fixed stimulation amplitude for each electrode. In this work, we look at the feasibility of using a spiking neural network (SNN) instead of the ANN allowing the model to provide time-varying stimulation patterns. During training, we mapped pulses of the SNN through a phosphene simulator which models the perceptual response. Results on the MNIST dataset show that our SNN-based encoder demonstrates reasonable reconstruction quality compared to a state-of-the-art ANN while requiring 2× fewer operations (1.16 vs 2.44 GFLOPs). Preliminary results show that the network trained on N-MNIST, a dataset of spikes from the spiking retina event camera on MNIST images, shows successful reconstructed recognizable digit shapes, suggesting the possibility of using event cameras for visual prosthesis.