Facial Expression Metrics as Digital Biomarkers of Neurologic Disease

Facial movements can be key indicators of neurological health and emotional state, offering insights into motor and neuropsychiatric functions that are disrupted in neurologic disorders. Neurological disease can present with characteristic differences in facial movements, like the masked facies of parkinsonism. Automated digital facial expression recognition could assist in asynchronous, remote and objective diagnostic processes. We hypothesized that facial movements relating to smiling, frowning, and blinking could be extracted from brief video-taped encounters in a clinic setting and used to (1) differentiate between neurologic diagnoses, and (2) identify people with symptoms of anxiety and depression.

Using untargeted recruitment, individuals with multiple sclerosis (MS), other conditions (parkinsonism, frontotemporal dementia (FTD)), and healthy controls (HC) enrolled in an ongoing digital phenotyping study. Participant faces were video-recorded during a spontaneous language task. Videos were processed using OpenFace 2.2.0, an open-access digital tool pre-trained for facial landmark detection and facial action unit recognition. Participants with MS completed the General Anxiety Disorder-7 (GAD-7) and the Hospital Anxiety and Depression Scale (HADS-D).

Videos were analyzed for adults with MS (n=151, mean age 48, 72% female), parkinsonism (n=23, mean age 67, 35% female), FTD (n=14, mean age 68, 29% female), and HCs (n=33, mean age 55, 58% female). Sampling duration was 60-90 seconds; 91% videos passed quality control. Individuals with parkinsonism had decreased eye-blinking compared to all groups, and decreased smiling and increased brow-lowering compared to MS and HCs. Individuals with FTD had increased blinking relative to other groups. There were no significant differences between individuals with MS and HCs. Classification accuracy for partition analysis model was 88% (ROC-AUC 0.84 for parkinsonism). In individuals with MS, decreased variability in brow lowerering was seen with higher anxiety symptoms, and decreased cheek raising intensity was seen with higher depression symptoms.

Digitally identified facial movements have face validity for recapitulating known clinical characteristics of neurological disease, as well as reflecting internal state relating to mood. This provides a foundation for expanded longitudinal validation of computer vision-based facial movement analysis in neurological research. However, findings should be interpreted in the context of sample size imbalance across diagnostic groups, which may have influenced classification performance.