Prior expectations guide multisensory integration during face-to-face communication.

Area of Science:

• Cognitive Science
• Neuroscience
• Computational Linguistics

Background:

• Face-to-face communication relies on integrating multisensory signals like voice and gaze.
• The computational basis for binding and segregating these signals during conversation is poorly understood.
• Bayesian causal inference offers a framework to model how the brain combines sensory information.

Purpose of the Study:

• To investigate if prior expectations influence the integration and segregation of vocal and bodily signals in communication.
• To determine if audiovisual signals with congruent communicative intent are preferentially integrated.
• To assess whether causal inference in this context follows optimal Bayesian strategies or approximate heuristics.

Main Methods:

• A spatial localization task using audiovisual clips with varying spatial disparities between voice and visual cues.
• Manipulation of pragmatic correspondence by presenting communicative (addressed) versus non-communicative (unaddressed) stimuli.
• Measurement of audiovisual integration using the ventriloquist effect and computational modeling.

Main Results:

• Observers employed approximate heuristics for audiovisual causal inference, closely matching optimal Bayesian predictions.
• A stronger tendency to integrate vocal and bodily information was observed when signals conveyed congruent communicative intent.
• Pragmatic correspondences were shown to enhance multisensory integration.

Conclusions:

• Face-to-face communication is shaped by expectations about how multisensory signals should be interpreted.
• Pragmatic factors significantly influence the integration of auditory and visual cues in communication.
• The brain uses approximate Bayesian inference to efficiently process communicative signals.