Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Hearing01:31

Hearing

When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.
Auditory Perception01:17

Auditory Perception

The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the cochlea, a...
The Cochlea01:13

The Cochlea

The cochlea is a coiled structure in the inner ear that contains hair cells—the sensory receptors of the auditory system. Sound waves are transmitted to the cochlea by small bones attached to the eardrum called the ossicles, which vibrate the oval window that leads to the inner ear. This causes fluid in the chambers of the cochlea to move, vibrating the basilar membrane.
Facial Feedback Hypothesis01:24

Facial Feedback Hypothesis

Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role of...
Perceiving Loudness, Pitch, and Location01:21

Perceiving Loudness, Pitch, and Location

The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by identifying...
Auditory Pathway01:15

Auditory Pathway

Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking the...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

ERP data on auditory imagery of native and non-native English speech during silent reading.

Data in brief·2020
Same author

The time-course of cortical responses to speech revealed by fast optical imaging.

Brain and language·2018
Same author

Age-Related Changes in Temporal and Spectral Cue Weights in Speech.

Language and speech·2017
Same author

List Equivalency of PRESTO for the Evaluation of Speech Recognition.

Journal of the American Academy of Audiology·2015
Same author

Read My Lips: Brain Dynamics Associated with Audiovisual Integration and Deviance Detection.

Journal of cognitive neuroscience·2015
Same author

Influence of early linguistic experience on regional dialect categorization by an adult cochlear implant user: a case study.

Ear and hearing·2014

Related Experiment Video

Updated: May 17, 2026

Infant Auditory Processing and Event-related Brain Oscillations
06:34

Infant Auditory Processing and Event-related Brain Oscillations

Published on: July 1, 2015

Seeing facial motion affects auditory processing in noise.

Jaimie L Gilbert1, Charissa R Lansing, Susan M Garnsey

  • 1University of Illinois, Urbana-Champaign, IL, USA. jaimie.gilbert@gmail.com

Attention, Perception & Psychophysics
|October 17, 2012
PubMed
Summary
This summary is machine-generated.

Observing a speaker's dynamic phonetic gestures significantly improves speech understanding in noise. This visual cueing enhances auditory processing, particularly for the N1 event-related potential, aiding speech perception.

More Related Videos

Behavioral Assessment of Hearing in 2 to 4 Year-old Children: A Two-interval, Observer-based Procedure Using Conditioned Play-based Responses
14:05

Behavioral Assessment of Hearing in 2 to 4 Year-old Children: A Two-interval, Observer-based Procedure Using Conditioned Play-based Responses

Published on: January 23, 2017

Somatosensory Event-related Potentials from Orofacial Skin Stretch Stimulation
06:56

Somatosensory Event-related Potentials from Orofacial Skin Stretch Stimulation

Published on: December 18, 2015

Related Experiment Videos

Last Updated: May 17, 2026

Infant Auditory Processing and Event-related Brain Oscillations
06:34

Infant Auditory Processing and Event-related Brain Oscillations

Published on: July 1, 2015

Behavioral Assessment of Hearing in 2 to 4 Year-old Children: A Two-interval, Observer-based Procedure Using Conditioned Play-based Responses
14:05

Behavioral Assessment of Hearing in 2 to 4 Year-old Children: A Two-interval, Observer-based Procedure Using Conditioned Play-based Responses

Published on: January 23, 2017

Somatosensory Event-related Potentials from Orofacial Skin Stretch Stimulation
06:56

Somatosensory Event-related Potentials from Orofacial Skin Stretch Stimulation

Published on: December 18, 2015

Area of Science:

  • Auditory Neuroscience
  • Speech Perception
  • Visual Speech Processing

Background:

  • Speech perception in noise is crucial for communication.
  • Visual cues from speech production can enhance auditory perception.
  • Conflicting evidence exists regarding the specific visual elements that aid speech perception.

Purpose of the Study:

  • To investigate the relative contributions of audibility and visual cue type on speech perception in noise.
  • To determine which visual aspects of speech production enhance understanding and auditory processing.

Main Methods:

  • Three experiments were conducted with young adults with normal hearing and vision.
  • Participants' speech understanding, response times, and auditory N1 event-related potentials were measured.
  • Static visual cues, dynamic facial motion, and linguistic phonetic gestures were presented.

Main Results:

  • Phonetic gestures in speech production significantly improved speech understanding in noise, reducing response times and increasing sensitivity.
  • Dynamic facial motion decreased N1 latency, while meaningful linguistic gestures reduced N1 amplitude.
  • Auditory-visual facilitation was partially supported as distinct to natural, dynamic speech gestures.

Conclusions:

  • Dynamic, linguistically relevant visual cues from speech production are critical for enhancing speech perception in noisy environments.
  • Visual speech information modulates early auditory processing, as evidenced by changes in N1 event-related potentials.
  • Understanding the interplay between visual and auditory information is key to improving speech perception technologies.