Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Association Areas of the Cortex01:21

Association Areas of the Cortex

6.8K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
6.8K
Auditory Perception01:17

Auditory Perception

657
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
657
Perceiving Loudness, Pitch, and Location01:21

Perceiving Loudness, Pitch, and Location

536
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
536
Facial Feedback Hypothesis01:24

Facial Feedback Hypothesis

305
Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role...
305
Auditory Pathway01:15

Auditory Pathway

6.0K
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
6.0K
The Anchoring-and-Adjustment Heuristic01:25

The Anchoring-and-Adjustment Heuristic

7.5K
In order to make good decisions, we use our knowledge and our reasoning. Often, this knowledge and reasoning is sound and solid. However, sometimes, we are swayed by biases or by others manipulating a situation. For example, let’s say you and three friends wanted to rent a house and had a combined target budget of $1,600. The realtor shows you only very run-down houses for $1,600 and then shows you a very nice house for $2,000. Might you ask each person to pay more in rent to get the...
7.5K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Modulation masking and fine structure shape neural envelope coding to predict speech intelligibility across diverse listening conditions.

The Journal of the Acoustical Society of America·2021
See all related articles

Related Experiment Video

Updated: Oct 15, 2025

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
09:09

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

584

Spatial alignment between faces and voices improves selective attention to audio-visual speech.

Justin T Fleming1, Ross K Maddox2, Barbara G Shinn-Cunningham3

  • 1Speech and Hearing Bioscience and Technology Program, Harvard University, 243 Charles Street, Boston, Massachusetts 02114, USA.

The Journal of the Acoustical Society of America
|October 31, 2021
PubMed
Summary
This summary is machine-generated.

Spatial alignment of faces and voices enhances speech understanding in noisy, multi-talker settings. This visual-auditory (AV) spatial alignment is crucial for effectively focusing on a target speaker amidst distractions.

More Related Videos

Mapping Cortical Dynamics Using Simultaneous MEG/EEG and Anatomically-constrained Minimum-norm Estimates: an Auditory Attention Example
08:45

Mapping Cortical Dynamics Using Simultaneous MEG/EEG and Anatomically-constrained Minimum-norm Estimates: an Auditory Attention Example

Published on: October 24, 2012

14.8K
fMRI Mapping of Brain Activity Associated with the Vocal Production of Consonant and Dissonant Intervals
11:15

fMRI Mapping of Brain Activity Associated with the Vocal Production of Consonant and Dissonant Intervals

Published on: May 23, 2017

7.3K

Related Experiment Videos

Last Updated: Oct 15, 2025

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
09:09

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

584
Mapping Cortical Dynamics Using Simultaneous MEG/EEG and Anatomically-constrained Minimum-norm Estimates: an Auditory Attention Example
08:45

Mapping Cortical Dynamics Using Simultaneous MEG/EEG and Anatomically-constrained Minimum-norm Estimates: an Auditory Attention Example

Published on: October 24, 2012

14.8K
fMRI Mapping of Brain Activity Associated with the Vocal Production of Consonant and Dissonant Intervals
11:15

fMRI Mapping of Brain Activity Associated with the Vocal Production of Consonant and Dissonant Intervals

Published on: May 23, 2017

7.3K

Area of Science:

  • Auditory Perception
  • Visual Perception
  • Human-Computer Interaction

Background:

  • Visual cues, such as seeing a talker's face, significantly improve speech intelligibility in noisy environments when audio-visual signals are temporally synchronized.
  • The role of spatial alignment between faces and voices in multi-talker scenarios remains less understood.

Purpose of the Study:

  • To investigate the impact of spatial alignment between faces and voices on selective attention to speech in noise.
  • To determine if spatial alignment is a critical factor for audio-visual (AV) speech perception in complex listening environments.

Main Methods:

  • Conducted online experiments using a selective attention task in multi-talker noise.
  • Manipulated spatial alignment of faces and voices (same vs. different hemifield) and signal-to-noise ratio (SNR).
  • Investigated the influence of eye gaze direction on performance in auditory-only conditions.

Main Results:

  • Task performance improved when faces and voices were spatially aligned (same hemifield).
  • Spatial misalignment between audio and visual speech signals incurred a performance cost.
  • The effect of AV spatial alignment was more pronounced at lower SNRs, though lipreading accuracy introduced a floor effect.

Conclusions:

  • Spatial alignment between faces and voices is a significant factor contributing to the ability to selectively attend to audio-visual (AV) speech.
  • Effective AV speech perception in noisy, multi-talker environments relies on both temporal and spatial congruence of speech cues.