Audiovisual speech from emotionally expressive and lateralized faces

Area of Science:

• Cognitive Neuroscience
• Psychology
• Speech Perception

Background:

• Emotional expression influences audiovisual speech perception.
• Facial asymmetries may play a role in processing affective and speech information.

Purpose of the Study:

• To investigate the interaction between facial emotional expression, lateralization, and audiovisual speech perception.
• To determine if facial asymmetries affect the processing of emotional and speech cues.

Main Methods:

• Experiments compared perception of sentences with happy, sad, and neutral emotions.
• Dynamic facial chimeras exaggerated asymmetrical differences between left and right face halves.
• Independent testing of left and right face sides isolated affective and speech processing asymmetries.

Main Results:

• The right side of the face and right-face chimeras showed superior speech perception compared to the left.
• Happy facial expressions improved speech performance bilaterally.
• Sad emotions impaired visual speech perception, particularly from the left face side.

Conclusions:

• Facial asymmetries impact audiovisual speech perception, with the right face being more effective.
• Emotional valence critically modulates speech perception, with positive emotions facilitating and negative emotions inhibiting.
• Approach-related information conveyed by facial expressions may enhance audiovisual speech detection.