The cross-race effect in automatic facial expression recognition violates measurement invariance

Area of Science:

• Psychology
• Cognitive Neuroscience
• Computer Vision

Background:

• Facial emotion recognition (FER) research increasingly uses automatic methods.
• Automatic FER offers objective, rapid analysis of large datasets.
• Concerns exist regarding the generalizability of models trained on Western faces to other populations.

Purpose of the Study:

• To perform a cross-racial validation of popular facial emotion recognition systems (FaceReader, DeepFace).
• To assess the performance of these systems on Western and Eastern facial datasets.
• To investigate potential biases in automatic emotion recognition across cultures.

Main Methods:

• Utilized two popular automatic facial emotion recognition systems: FaceReader and DeepFace.
• Employed two Western and two Eastern face datasets for cross-racial validation.
• Evaluated system accuracy in recognizing emotion categories.

Main Results:

• Both systems achieved high accuracy on Western datasets but performed poorly on Eastern datasets.
• Recognition accuracy for negative emotions was particularly reduced on Eastern faces.
• Measurements of happiness were found to be accurate and invariant across racial groups.

Conclusions:

• Automatic FER systems may exhibit racial bias, cautioning their use in cross-cultural studies involving non-Western faces.
• Despite limitations, happiness measurements from these systems are reliable for cross-cultural positive psychology research.
• Future research should focus on developing more culturally invariant FER models.