Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Higher Mental Functions of the Brain: Language01:10

Higher Mental Functions of the Brain: Language

Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language and Cognition01:27

Language and Cognition

Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

The neural basis of laughter.

Trends in neurosciences·2026
Same author

Human detection of AI-generated faces and voices is not domain-general.

Scientific reports·2026
Same author

Dopaminergic manipulation modulates frequency-specific EEG connectivity patterns: evidence from a single dose drug challenge study.

Frontiers in neuroscience·2026
Same author

Implicit and Explicit Voice Training Effects on Speech-on-Speech Perception and Listening Effort.

Ear and hearing·2026
Same author

Self-recognition shapes evaluations of (self-) voice attractiveness.

Cognition·2026
Same author

Executive resources shape the impact of language predictability across the adult lifespan.

eLife·2026

Related Experiment Video

Updated: May 25, 2026

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

Speech comprehension aided by multiple modalities: behavioural and neural interactions.

Carolyn McGettigan1, Andrew Faulkner, Irene Altarelli

  • 1Institute of Cognitive Neuroscience, University College London, 17 Queen Square, London WC1N 3AR, UK. c.mcgettigan@ucl.ac.uk

Neuropsychologia
|January 24, 2012
PubMed
Summary
This summary is machine-generated.

Understanding speech involves integrating auditory, visual, and linguistic cues. Brain regions like the posterior superior temporal sulcus (pSTS) and fusiform gyrus are key for combining these signals, while frontal and temporal areas support successful speech comprehension.

More Related Videos

A Novel Experimental and Analytical Approach to the Multimodal Neural Decoding of Intent During Social Interaction in Freely-behaving Human Infants
11:14

A Novel Experimental and Analytical Approach to the Multimodal Neural Decoding of Intent During Social Interaction in Freely-behaving Human Infants

Published on: October 4, 2015

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks
08:32

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks

Published on: September 5, 2019

Related Experiment Videos

Last Updated: May 25, 2026

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

A Novel Experimental and Analytical Approach to the Multimodal Neural Decoding of Intent During Social Interaction in Freely-behaving Human Infants
11:14

A Novel Experimental and Analytical Approach to the Multimodal Neural Decoding of Intent During Social Interaction in Freely-behaving Human Infants

Published on: October 4, 2015

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks
08:32

Examining Online Syntactic Processing of Spoken Complex Sentences in Chinese Using Dual-Modal Interference Tasks

Published on: September 5, 2019

Area of Science:

  • Neuroscience
  • Cognitive Science
  • Psycholinguistics

Background:

  • Speech comprehension is a complex cognitive process relying on multisensory integration.
  • Effective communication necessitates combining auditory, visual (e.g., facial expressions, gestures), and linguistic information.
  • Understanding the neural mechanisms underlying this integration is crucial for deciphering speech perception.

Purpose of the Study:

  • To investigate the neural interactions between auditory, visual, and linguistic information during speech comprehension.
  • To identify brain regions responsive to individual sensory inputs and their combined effects.
  • To correlate neural activity with behavioral measures of speech intelligibility and comprehension performance.

Main Methods:

  • Functional magnetic resonance imaging (fMRI) was employed to observe brain activity.
  • Participants passively viewed videos of spoken sentences with manipulated auditory clarity (noise-vocoding), visual clarity (Gaussian blurring), and linguistic predictability.
  • Behavioral experiments were conducted to assess stimulus intelligibility and individual comprehension abilities.

Main Results:

  • Increased auditory and visual clarity enhanced neural signals in overlapping regions of the posterior superior temporal sulcus (pSTS).
  • Interactions between auditory-visual and auditory-linguistic factors were observed outside temporal cortex, with peak responses at intermediate auditory clarity.
  • Neural activation in dorsolateral temporal cortex, inferior frontal cortex, and left fusiform gyrus correlated with overall stimulus intelligibility.
  • Posterior STS and fusiform gyrus showed enhanced responses to visual and linguistic information at intermediate auditory clarity.
  • Higher individual comprehension performance was linked to increased activation in the left inferior frontal gyrus and left posterior STS.

Conclusions:

  • A widespread neural network supports multimodal speech comprehension.
  • The posterior STS and fusiform gyrus serve as crucial convergence zones for auditory, visual, and linguistic information.
  • Left-dominant temporal and frontal cortical regions are vital for successful speech comprehension, particularly when integrating multiple information streams.