Related Experiment Video
Updated: Jun 5, 2026

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
The Role of Facial Action Units in Investigating Facial Movements During Speech
Aliya A Newby1, Ambika Bhatta1, Charles Kirkland1
1Biomedical Engineering Program, Center for Biomechanical & Rehabilitation Engineering, School of Engineering and Applied Sciences, University of the District of Columbia, 4200 Connecticut Ave. NW, Washington, DC 20008, USA.
Abstract:
Investigating how facial movements can be used to characterize and quantify speech is important, in particular, to aid those suffering from motor control speech disorders. Here, we sought to investigate how facial action units (AUs), previously used to classify human expressions and emotion, could be used to quantify and understand unimpaired human speech. Fourteen (14) adult participants (30.1 ± 7.9 years old), fluent in English, with no speech impairments, were examined. Within each data collection session, 6 video trials per participant per phoneme were acquired (i.e., 102 trials total/phoneme). The participants were asked to vocalize the vowels /æ/, /ɛ/, /ɪ/, /ɒ/, and /ʊ/; the consonants /b/, /n/, /m/, /p/, /h/, /w/, and /d/; and the diphthongs /eI/, /ʌɪ/, /i/, /a:/, and /u:/. Through the use of Python Py-Feat, our analysis displayed the AU contributions for each phoneme. The important implication of our methodological findings is that AUs could be used to quantify speech in populations with no speech disability; this has the potential to be broadened toward providing feedback and characterization of speech changes and improvements in impaired populations. This would be of interest to persons with speech disorders, speech language pathologists, engineers, and physicians.
Related Concept Videos
Association Areas of the Cortex
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Facial Feedback Hypothesis
Muscles for Facial Expressions
Non-Verbal Cues
