Facial expression is retained in deep networks trained for face identification.

Area of Science:

• Computer Vision
• Cognitive Neuroscience
• Machine Learning

Background:

• Facial expressions complicate 2D image identification.
• Brain systems must separate visual cues for social interaction.
• Deep convolutional neural networks (DCNNs) trained for face identification retain identity-irrelevant information like viewpoint.

Purpose of the Study:

• Investigate if DCNNs trained for identity recognition also retain expression information.
• Determine if this expression information generalizes across different viewpoints.
• Explore how DCNNs represent identity, expression, and viewpoint.

Main Methods:

• Generated DCNN representations for a controlled dataset of faces with varying expressions and viewpoints.
• Utilized 2D visualizations to analyze hierarchical groupings in DCNN representations.
• Applied linear discriminant analysis to classify facial expressions from DCNN representations.

Main Results:

• DCNN representations hierarchically grouped by identity, then viewpoint, then expression.
• Accurate expression classification (mean 76.8%) was achieved, generalizing across viewpoints.
• Representational similarity analysis showed viewpoint influenced within-identity image similarity more than expression.

Conclusions:

• Identity-trained DCNNs retain shape-deformable information about expression and viewpoint.
• This unified representation aligns with hypotheses of ventral visual stream processing.
• DCNNs offer a model for understanding how the brain integrates facial features for social cognition.