Related Experiment Video
Updated: Jun 12, 2025

Cross-Modal Multivariate Pattern Analysis
Published on: November 9, 2011
From unimodal to multimodal dynamics of verbal and nonverbal cues during unstructured conversation
Tifenn Fauviaux1, Ludovic Marin1, Mathilde Parisi1
1EuroMov Digital Health in Motion, Univ Montpellier, IMT Mines Ales, Montpellier, France.
Abstract:
Conversations encompass continuous exchanges of verbal and nonverbal information. Previous research has demonstrated that gestures dynamically entrain each other and that speakers tend to align their vocal properties. While gesture and speech are known to synchronize at the intrapersonal level, few studies have investigated the multimodal dynamics of gesture/speech between individuals. The present study aims to extend our comprehension of unimodal dynamics of speech and gesture to multimodal speech/gesture dynamics. We used an online dataset of 14 dyads engaged in unstructured conversation. Speech and gesture synchronization was measured with cross-wavelets at different timescales. Results supported previous research on intrapersonal speech/gesture coordination, finding synchronization at all timescales of the conversation. Extending the literature, we also found interpersonal synchronization between speech and gesture. Given that the unimodal and multimodal synchronization occurred at similar timescales, we suggest that synchronization likely depends on the vocal channel, particularly on the turn-taking dynamics of the conversation.
More Related Videos
11:14A Novel Experimental and Analytical Approach to the Multimodal Neural Decoding of Intent During Social Interaction in Freely-behaving Human Infants
Published on: October 4, 2015
08:36Dynamic Inter-subject Functional Connectivity Reveals Moment-to-Moment Brain Network Configurations Driven by Continuous or Communication Paradigms
Published on: March 21, 2019
Related Concept Videos
Relationship Formation
Therapeutic Communication
Verbal communication depends on language or a prescribed way of using words so that people can share information effectively. The critical aspects of verbal...
Levels of Communication II: Organizational, Public, and Group Dynamics
Language and Cognition
Nonconscious Mimicry
Barriers to Effective Communication II
Cultural barriers:
Differences in values, beliefs, religion, knowledge, and tradition can significantly impact communication. Awareness of nonverbal cues is critical, especially when conversing with a patient from a different culture. What appears appropriate in one culture may be inappropriate in another.
Semantic barriers:
As a result of their tendency to use...