Facial Motion Analysis beyond Emotional Expressions

Area of Science:

• Computer Science
• Artificial Intelligence
• Linguistics

Background:

• Facial motion analysis has advanced significantly, yet primarily focuses on basic emotions, neglecting non-verbal communication signals.
• Grammatical Facial Expressions (GFEs) are crucial in sign languages and expressive spoken language but are understudied.

Purpose of the Study:

• To focus on the classification of Grammatical Facial Expressions (GFEs) vital for sign language and expressive speech.
• To investigate the effectiveness of deep learning models for GFE recognition using various input features.

Main Methods:

• Collected a new dataset (LSE_GFE) of Spanish Sign Language sentences.
• Extracted intervals for three GFE types: negation, closed queries, and open queries.
• Evaluated multiple deep learning models, including Graph Convolutional Networks (GCNs), using face landmarks as input on LSE_GFE and BUHMAP datasets.

Main Results:

• Deep learning models, particularly GCNs, demonstrated reliable learning of GFEs.
• Input features based solely on face landmarks proved sufficient for accurate GFE classification.
• The study validates the potential of GCNs for analyzing subtle facial movements in communication.

Conclusions:

• Grammatical Facial Expressions can be reliably learned using Graph Convolutional Networks fed with face landmarks.
• This research opens new avenues for analyzing non-verbal communication in both sign and spoken languages.
• The findings support the development of more sophisticated facial motion analysis systems for diverse linguistic applications.