Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Auditory Perception01:17

Auditory Perception

1.5K
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
1.5K
Perceiving Loudness, Pitch, and Location01:21

Perceiving Loudness, Pitch, and Location

1.3K
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
1.3K
Perception of Sound Waves01:01

Perception of Sound Waves

6.1K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
6.1K
Constraints and Statical Determinacy01:26

Constraints and Statical Determinacy

1.1K
In structural engineering, the equilibrium of a system is not only determined by its equations of equilibrium but also with the help of constraints. Constraints refer to restrictions on the motion of a system. The proper combinations of constraints can minimize the total number of constraints needed to maintain a system in mechanical equilibrium. When this happens, the system is said to be statically determinate. For such systems, the unknown reaction supports can be estimated using equilibrium...
1.1K
Facial Feedback Hypothesis01:24

Facial Feedback Hypothesis

885
Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role...
885
Factors Affecting Perception01:25

Factors Affecting Perception

3.2K
Perception is influenced by perceptual set, context, motivation, and emotion. Perceptual set, or perceptual expectancy, refers to the tendency to perceive things in a particular way, influenced by previous experiences and expectations. This phenomenon affects the interpretation of stimuli, creating a set of mental tendencies and assumptions that impact sensory perceptions of sound, taste, touch, and sight.
An illustrative example of a perceptual set is the scenario where an airline pilot told...
3.2K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Investigating the analytical robustness of the social and behavioural sciences.

Nature·2026
Same author

Measuring brain sensitivity to semantic distance in spoken narrative comprehension.

Cortex; a journal devoted to the study of the nervous system and behavior·2026
Same author

Cortical connectivity in speech and sensory networks associated with childrens' listening and attention disorders.

medRxiv : the preprint server for health sciences·2025
Same author

A high-density diffuse optical tomography dataset of naturalistic viewing.

Scientific data·2025
Same author

Does Training with Structured Input Improve Perception and Comprehension of Bound Morphemes in Children Who Are Deaf or Hard-of-Hearing?

Seminars in hearing·2025
Same author

Publisher Correction: fNIRS reproducibility varies with data quality, analysis pipelines, and researcher experience.

Communications biology·2025

Related Experiment Video

Updated: Apr 14, 2026

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
09:09

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

1.0K

Prediction and constraint in audiovisual speech perception.

Jonathan E Peelle1, Mitchell S Sommers2

  • 1Department of Otolaryngology, Washington University in St. Louis, St. Louis, MO, USA.

Cortex; a Journal Devoted to the Study of the Nervous System and Behavior
|April 20, 2015
PubMed
Summary

Visual speech cues enhance auditory processing, improving speech recognition, especially in noisy environments. This multisensory integration reduces cognitive load and aids lexical selection through early and late processing stages.

Keywords:
Audiovisual speechMultisensory integrationPredictive codingPredictive timingSpeech perception

More Related Videos

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language
09:27

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

Published on: October 13, 2018

10.9K
Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
07:36

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects

Published on: November 30, 2018

16.6K

Related Experiment Videos

Last Updated: Apr 14, 2026

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
09:09

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

1.0K
Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language
09:27

Using Eye Movements Recorded in the Visual World Paradigm to Explore the Online Processing of Spoken Language

Published on: October 13, 2018

10.9K
Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
07:36

Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects

Published on: November 30, 2018

16.6K

Area of Science:

  • Neuroscience
  • Cognitive Science
  • Speech Processing

Background:

  • Face-to-face speech requires processing complex multisensory information.
  • Visual speech provides crucial timing and articulation cues complementing auditory signals.
  • Listeners benefit from visual speech, particularly in challenging listening conditions like background noise.

Purpose of the Study:

  • To review behavioral and neurophysiological evidence on how listeners utilize visual speech information.
  • To explore the mechanisms of multisensory integration in audiovisual speech perception.
  • To understand the role of visual cues in enhancing speech recognition and reducing cognitive load.

Main Methods:

  • Review of behavioral studies on speech perception.
  • Analysis of neurophysiological evidence, including electrophysiology and neuroimaging.
  • Examination of oscillatory cortical entrainment and brain activity in auditory and heteromodal regions.

Main Results:

  • Multisensory integration of audiovisual speech cues significantly improves recognition accuracy, especially in noise.
  • Visual speech enhances auditory processing by increasing sensitivity to acoustic cues and aiding prediction.
  • Neuroimaging reveals enhanced activity in auditory cortex and involvement of the posterior superior temporal sulcus in integrative processing.

Conclusions:

  • Audiovisual speech perception involves multiple stages of integration, supported by distinct neuroanatomical mechanisms.
  • Visual speech information influences auditory processing, improving sensitivity and constraining lexical selection.
  • Successful speech perception relies on integrating information from both auditory and visual modalities.