Exploration of Mehrabian's communication model with an android

Area of Science:

• Human-Computer Interaction
• Affective Computing
• Social Psychology

Background:

• Multimodal emotional expressions are crucial for effective interpersonal communication.
• Previous research, like Mehrabian's model, suggested a hierarchy of emotional impact from facial, vocal, and verbal cues, but lacked comprehensive experimental validation in face-to-face settings.
• Limitations in human acting capabilities have hindered studies examining all three emotional expression modalities simultaneously.

Purpose of the Study:

• To experimentally investigate the relative emotional impact of facial, vocal, and verbal expressions in a controlled face-to-face scenario.
• To address the limitations of human acting by utilizing an android for consistent emotional expression generation.
• To validate Mehrabian's model regarding the hierarchy of emotional impact across different communication modalities.

Main Methods:

• An android (Nikola) was programmed to systematically display distinct facial, vocal, and verbal expressions of positive, neutral, and negative emotions.
• Participants engaged in face-to-face interactions with the android and rated the emotional valence of its expressions.
• Comparative experiments were conducted using human raters and ChatGPT to assess unimodal emotional valence.

Main Results:

• Facial expressions were ranked as having the highest emotional impact, followed by vocal expressions, and then verbal expressions.
• The android's systematically varied expressions allowed for a clear ranking of emotional impact across modalities.
• Unimodal expressions of facial, vocal, and verbal cues showed comparable emotional valence when assessed by human raters and ChatGPT.

Conclusions:

• The study provides the first experimental evidence validating Mehrabian's model in a face-to-face context.
• Facial (nonverbal) expressions hold the most significant emotional impact in face-to-face communication.
• Androids can serve as effective tools for studying human emotional expression and communication dynamics.