Decoding multi-joint hand movements from brain signals by learning a synergy-based neural manifold

Brain-computer interfaces have shown great potential in the reconstruction of motor functions. However, decoding complex and natural movements, such as hand movements, remains challenging. Traditional approaches primarily decode the movement of multiple joints in the hand independently, while the inherent synergies underlying these movements have not been well explored. Here, we demonstrate that complex hand movements can be decomposed into a set of motor primitives, each involving a synergy of multi-joint movements. Motor cortical neural activities recruit the motor synergies through spatiotemporal parameters to accomplish the complex motor targets. By learning a joint neural-motor representation of these motor synergies and decoding spatiotemporal parameters rather than the joint-level kinematics, significant improvement could be obtained in hand movement decoding. We propose a neural decoding framework, SynergyNet, to effectively learn the neural-motor synergies for hand movement control. The proposed approach significantly outperforms benchmark methods and provides high interpretability with the hand movement neural decoding task.