Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Hearing

Hearing

When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.

Language and Cognition

Language and Cognition

Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.

Higher Mental Functions of the Brain: Language

Higher Mental Functions of the Brain: Language

Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...

Auditory Pathway

Auditory Pathway

Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...

Perceiving Loudness, Pitch, and Location

Perceiving Loudness, Pitch, and Location

The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...

Parallel Processing

Parallel Processing

The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

A Review of the Challenge of Pre-Existing Humoral Immunity in Adeno-Associated Virus Gene Therapy and Potential Solutions.

Human gene therapy·2026

Same author

Acoustic parameter combinations underlying mapping of pseudoword sounds to multiple domains of meaning: Representational similarity analyses and machine-learning models.

The Journal of the Acoustical Society of America·2025

Same author

A randomized trial showing mnemonic strategy training increases memory, brain activation, and functional connectivity more than vanishing cue training in cognitively intact older adults.

Neuropsychological rehabilitation·2025

Same author

ACOUSTIC PARAMETER COMBINATIONS UNDERLYING MAPPING OF AUDITORY PSEUDOWORD SOUNDS TO MULTIPLE DOMAINS OF MEANING: A MACHINE LEARNING APPROACH.

bioRxiv : the preprint server for biology·2025

Same author

Phonetic convergence enhances speech intelligibility.

Attention, perception & psychophysics·2025

Same author

DIFFERENTIAL CONTRIBUTIONS OF THE SPECTRO-TEMPORAL AND VOCAL CHARACTERISTICS OF AUDITORY PSEUDOWORDS TO MULTIPLE SOUND-SYMBOLIC MAPPINGS.

bioRxiv : the preprint server for biology·2024

Search research articles

Related Experiment Video

Updated: Jul 5, 2025

Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder

Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder

Published on: April 22, 2015

Multisensory and lexical information in speech perception.

Josh Dorsi¹, Simon Lacey^1,2,3, K Sathian^1,2,3

¹Department of Neurology, Penn State College of Medicine, Hershey, PA, United States.

Frontiers in Human Neuroscience

|January 23, 2024

Summary

This summary is machine-generated.

Multisensory information, which involves input from various senses, is more fundamental to speech perception than lexical information (knowledge of words). This means our brains process non-auditory cues earlier and more critically for understanding speech.

Keywords:

audiovisual lexical multisensory sound symbolism speech perception

More Related Videos

Interaction between Phonological and Semantic Processes in Visual Word Recognition using Electrophysiology

Interaction between Phonological and Semantic Processes in Visual Word Recognition using Electrophysiology

Published on: June 29, 2021

Cross-Modal Multivariate Pattern Analysis

Cross-Modal Multivariate Pattern Analysis

Published on: November 9, 2011

Related Experiment Videos

Last Updated: Jul 5, 2025

Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder

Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder

Published on: April 22, 2015

Interaction between Phonological and Semantic Processes in Visual Word Recognition using Electrophysiology

Interaction between Phonological and Semantic Processes in Visual Word Recognition using Electrophysiology

Published on: June 29, 2021

Cross-Modal Multivariate Pattern Analysis

Cross-Modal Multivariate Pattern Analysis

Published on: November 9, 2011

Area of Science:

Cognitive Neuroscience
Auditory Perception
Speech Processing

Background:

Speech perception is influenced by both multisensory input and lexical knowledge.
The relative importance and fundamental nature of these two information sources remain debated.

Purpose of the Study:

To review existing literature on speech perception.
To argue that multisensory information plays a more fundamental role than lexical information.

Main Methods:

Literature review and synthesis of findings from reaction time studies.
Analysis of electroencephalographic (EEG) signal latencies.
Examination of how non-auditory sensory input affects phonetic perception.
Review of evidence for sound symbolism.

Main Results:

Multisensory information processing appears to occur earlier than lexical information processing, as indicated by reaction times and EEG latencies.
Non-auditory sensory input influences the perception of phonetic features, thereby shaping lexical processing.
Multisensory information contributes to the formation of lexical information through phenomena like sound symbolism.

Conclusions:

Multisensory information is more fundamental to speech perception than lexical information.
A revised framework for speech perception emphasizes the foundational role of multisensory input.
While both sources are influential, multisensory cues provide an earlier and more critical basis for understanding speech.