An efficient brain-heart coupling learning system for emotion recognition

In neuroscience, cardiac sympathetic-vagal nerve activity and central nervous system activity in the brain work in concert to maintain the body's response to emotional perception. Therefore, the efficient mining of brain-heart driving patterns in emotion perception is conducive to the development of emotion recognition system. However, the dynamic correlation between the brain and heart poses a challenge to accurately capture emotion related representations, which makes the understanding of emotion perception remain superficial. Here, we develop a neural-inspired emotional perception learning system (NEAL), which aims to mine the representative brain-heart coupling model to improve the understanding of emotional perception. Specifically, we first introduce a multi-kernel filter to generate the primary representation of semantic alignment. At the same time, the bilinear fusion module improves the model's understanding of brain-heart local dynamic correlation. In order to further capture the global brain-heart representations, a cross-perception mechanism based on dynamic routing is meticulously designed, and explored the response mode of brain-heart two-way communication to emotional perception. The results on Dreamer, MPED and MAHNOB-HCI datasets show that the proposed NEAL learning system has impressive performance advantages over the existing competitive methods. More importantly, this study has successfully improved the understanding of emotion perception and provided a new solution for further revealing the emotion-related neural activities.