Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Auditory Perception01:17

Auditory Perception

884
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
884
Perception of Sound Waves01:01

Perception of Sound Waves

5.3K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
5.3K
Perceiving Loudness, Pitch, and Location01:21

Perceiving Loudness, Pitch, and Location

806
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
806
Auditory Pathway01:15

Auditory Pathway

6.8K
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
6.8K
Hearing01:31

Hearing

56.1K
When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.
56.1K
Reconstruction of Signal using Interpolation01:10

Reconstruction of Signal using Interpolation

608
Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next...
608

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Nested mobile genetic elements mediating antimicrobial resistance genes mobility within and between <i>Acinetobacter</i> isolates.

iScience·2026
Same author

A prospective cohort study of the factors associated with recurrence in young patients with thyroid cancer.

Journal of the Endocrine Society·2026
Same author

The impact of mildly elevated HbA1c level, and low levels of high-density lipoprotein cholesterol and hemoglobin on microvascular angina.

Heart and vessels·2026
Same author

The Intrinsic Manifold of Spontaneous Activity Constrains Cortical Responses to Naturalistic Stimuli.

bioRxiv : the preprint server for biology·2026
Same author

Quantitative Analysis of the Effect of Gastrointestinal Water Dynamics on Oral Drug Absorption using a Physiologically Based Integrated Liquid and Intestinal Absorption Drug (ILIAD) Pharmacokinetic Model.

Pharmaceutical research·2026
Same author

Detection rate and maximum diameter of thyroid cysts among children and adolescents in thyroid ultrasound examination, The Fukushima Health Management Survey.

Endocrine journal·2026

Related Experiment Video

Updated: Dec 15, 2025

Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
11:39

Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique

Published on: September 7, 2022

2.4K

Vision perceptually restores auditory spectral dynamics in speech.

John Plass1,2, David Brang3, Satoru Suzuki2,4

  • 1Department of Psychology, University of Michigan, Ann Arbor, MI 48109; jplass@umich.edu.

Proceedings of the National Academy of Sciences of the United States of America
|July 8, 2020
PubMed
Summary
This summary is machine-generated.

Visual speech, specifically mouth movements, helps us hear better by providing detailed spectrotemporal information. This visual speech information aids auditory speech perception, especially when sound is degraded.

Keywords:
audiovisual speechmultisensoryspectrotemporalspeech perception

More Related Videos

Stimulus-specific Cortical Visual Evoked Potential Morphological Patterns
09:42

Stimulus-specific Cortical Visual Evoked Potential Morphological Patterns

Published on: May 12, 2019

6.3K
Dynamic Visual Tests to Identify and Quantify Visual Damage and Repair Following Demyelination in Optic Neuritis Patients
12:23

Dynamic Visual Tests to Identify and Quantify Visual Damage and Repair Following Demyelination in Optic Neuritis Patients

Published on: April 14, 2014

14.4K

Related Experiment Videos

Last Updated: Dec 15, 2025

Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
11:39

Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique

Published on: September 7, 2022

2.4K
Stimulus-specific Cortical Visual Evoked Potential Morphological Patterns
09:42

Stimulus-specific Cortical Visual Evoked Potential Morphological Patterns

Published on: May 12, 2019

6.3K
Dynamic Visual Tests to Identify and Quantify Visual Damage and Repair Following Demyelination in Optic Neuritis Patients
12:23

Dynamic Visual Tests to Identify and Quantify Visual Damage and Repair Following Demyelination in Optic Neuritis Patients

Published on: April 14, 2014

14.4K

Area of Science:

  • Auditory and Visual Perception
  • Speech Science
  • Cross-modal Interactions

Background:

  • Visual speech enhances auditory speech perception, but the specific visual cues and information conveyed remain unclear.
  • Existing models focus on low-level temporal cues or high-level abstract representations, neither fully explaining the richness of visual speech benefits.
  • The spectral content of speech is shaped by visible articulators, suggesting a potential link between visual and auditory speech features.

Purpose of the Study:

  • To investigate if the perceptual system exploits correlations between mid-level visual (oral deformations) and auditory (frequency modulations) speech features.
  • To determine if visual speech can provide detailed spectrotemporal information without high-level abstractions.
  • To assess how visual speech facilitates auditory speech comprehension under different degradation conditions.

Main Methods:

  • Predicting time-frequency dynamics of oral resonances (formants) from mouth shape during speech.
  • Evaluating the effect of isolated, speech-based shape deformations on perceptual sensitivity to frequency modulations.
  • Assessing visual speech facilitation of auditory sentence comprehension with spectrally or temporally degraded audio.

Main Results:

  • Oral resonance dynamics were precisely predictable from changing mouth shape.
  • Visual speech-based shape deformations improved perceptual sensitivity to frequency modulations.
  • Visual speech provided significantly greater comprehension enhancement for spectrally degraded auditory speech.

Conclusions:

  • The perceptual system likely exploits audiovisual correlations between oral deformations and frequency modulations to extract spectrotemporal information from visual speech.
  • This cross-modal correspondence allows for detailed information extraction without necessarily relying on high-level speech abstractions.
  • Visual speech offers condition-specific facilitation, particularly aiding in the recovery of auditory speech spectra.