Spontaneous facial expression in a small group can be automatically measured: an initial demonstration

Manual measurement of facial expression is labor intensive and difficult to standardize. Automated measurement seeks to address the need for valid, efficient, and reproducible measurement. Recent systems have shown promise in posed behavior and in structured contexts. Can automated measurement perform in more natural, less constrained settings? In the present study, previously unacquainted young adults sat around a circular table for 30 min of conversation. Video was selected for manual and automatic coding of Facial Action Coding System action units (AUs), examining, in particular, AU 6 (cheek raise) and AU 12 (lip corner pull), which together signal enjoyment. Moderate out-of-plane head motion and occlusion, which are challenging for automatic processing, were both common, as participants turned toward and away from each other or consumed drinks. Concurrent validity for both AUs was high. This is the first study to find that automated measurement of facial action in relatively unconstrained contexts can achieve results comparable with those of manual coding. Video demos of our software may be downloaded from http://brm.psychonomic-journals.org/content/supplemental.