Exploration of Mehrabian's communication model with an android

Multimodal emotional expressions play an essential role in real-life communication. Mehrabian and colleagues suggested that facial expressions may have the greatest emotional impact, followed by vocal and verbal expressions. However, no study has examined all three modalities in face-to-face situations in a single experiment, possibly due to limitations in human acting. We postulated that an android could be a useful solution to this problem. In this study, the android Nikola systematically changed its facial, vocal, and verbal expressions of negative, neutral, and positive emotions in a face-to-face situation. Participants rated the emotional valence of the expressions. The modalities were ranked from the greatest to the least emotional impact, as follows: facial expressions, then vocal expressions, and finally verbal expressions. Additional experiments with human raters and ChatGPT showed comparable emotional valence for facial, vocal, and verbal expressions presented unimodally. The results provide the first evidence validating Mehrabian's model, demonstrating the importance of facial or nonverbal expressions in face-to-face emotional communication.