Few-shot transfer learning for individualized braking intent detection on neuromorphic hardware

Objective.This work explores use of a few-shot transfer learning method to train and implement a convolutional spiking neural network (CSNN) on a BrainChip Akida AKD1000 neuromorphic system-on-chip for developing individual-level, instead of traditionally used group-level, models using electroencephalographic data. The efficacy of the method is studied on an advanced driver assist system related task of predicting braking intention.Approach.Data are collected from participants operating an NVIDIA JetBot on a testbed simulating urban streets for three different scenarios. Participants receive a braking indicator in the form of: (1) an audio countdown in a nominal baseline, stress-free environment; (2) an audio countdown in an environment with added elements of physical fatigue and active cognitive distraction; (3) a visual cue given through stoplights in a stress-free environment. These datasets are then used to develop individual-level models from group-level models using a few-shot transfer learning method, which involves: (1) creating a group-level model by training a CNN on group-level data followed by quantization and recouping any performance loss using quantization-aware retraining; (2) converting the CNN to be compatible with Akida AKD1000 processor; and (3) training the final decision layer on individual-level data subsets to create individual-customized models using an online Akida edge-learning algorithm.Main results.Efficacy of the above methodology to develop individual-specific braking intention predictive models by rapidly adapting the group-level model in as few as three training epochs while achieving at least 90% accuracy, true positive rate and true negative rate is presented. Further, results show the energy-efficiency of the neuromorphic hardware through a power reduction of over 97% with only a 1.3 × increase in latency when using the Akida AKD1000 processor for network inference compared to an Intel Xeon central processing unit. Similar results were obtained in a subsequent ablation study using a subset of five out of 19 channels.Significance.Especially relevant to real-time applications, this work presents an energy-efficient, few-shot transfer learning method that is implemented on a neuromorphic processor capable of training a CSNN as new data becomes available, operating conditions change, or to customize group-level models to yield personalized models unique to each individual.