Decoding multi-limb movements from two-photon calcium imaging of neuronal activity using deep learning

Objective.Brain-machine interfaces (BMIs) aim to restore sensorimotor function to individuals suffering from neural injury and disease. A critical step in implementing a BMI is to decode movement intention from recorded neural activity patterns in sensorimotor areas. Optical imaging, including two-photon (2p) calcium imaging, is an attractive approach for recording large-scale neural activity with high spatial resolution using a minimally-invasive technique. However, relating slow two-photon calcium imaging data to fast behaviors is challenging due to the relatively low optical imaging sampling rates. Nevertheless, neural activity recorded with 2p calcium imaging has been used to decode information about stereotyped single-limb movements and to control BMIs. Here, we expand upon prior work by applying deep learning to decode multi-limb movements of running mice from 2p calcium imaging data.Approach.We developed a recurrent encoder-decoder network (LSTM-encdec) in which the output is longer than the input.Main results.LSTM-encdec could accurately decode information about all four limbs (contralateral and ipsilateral front and hind limbs) from calcium imaging data recorded in a single cortical hemisphere.Significance.Our approach provides interpretability measures to validate decoding accuracy and expands the utility of BMIs by establishing the groundwork for control of multiple limbs. Our work contributes to the advancement of neural decoding techniques and the development of next-generation optical BMIs.