MultiFPE: Multi-Representation Fusion Network for Facial Palsy Evaluation in Facial Videos

Facial palsy is a common neuromuscular disorder that severely affects patients' physical and mental health. Facial palsy evaluation (FPE) is crucial for the formulation of facial functional treatment and rehabilitation. The traditional evaluation of facial palsy relies heavily on experience and subjective judgement of clinicians. However, the current development of artificial intelligence-based automated evaluation is challenged by the lack of benchmark results from publicly available datasets, and existing evaluation methods typically rely on a single facial representation while neglecting the potential information that can be gleaned from other representations. To address these issues, we first present reannotated versions of the public AFLFP and MEEI datasets, along with benchmark results using mainstream methods. Furthermore, we propose a novel multi-representation fusion network for FPE, named MultiFPE, which fully leverages fine-grained facial information from limited data to enhance the sensitivity to palsy-related facial discrepancies. Experimental results show that the proposed MultiFPE outperforms state-of-the-art approaches on two public datasets. The data and code are available at https://github.com/Yating180/MultiFPE.