Related Experiment Video
Updated: Aug 22, 2025
06:04
Systematic Hearing Performance Evaluation Process for Adolescents with Cochlear Implantation at Early Ages
Published on: March 24, 2023
452
Multisensory benefits for speech recognition in noisy environments.
Yonghee Oh1,2, Meg Schwalm2, Nicole Kalpin2
1Department of Otolaryngology-Head and Neck Surgery and Communicative Disorders, University of Louisville, Louisville, KY, United States.
Frontiers in Neuroscience
|November 7, 2022
Summary
Adding visual and tactile cues to auditory speech perception significantly improves word recognition in noise. Multimodal sensory integration, especially with both visual and tactile information, enhances speech understanding in challenging listening conditions.
Area of Science:
- Auditory perception
- Multisensory integration
- Speech processing
Background:
- Previous studies explored abstract visual cues for speech perception in noise.
- This research extends auditory-visual speech perception to the tactile domain.
Purpose of the Study:
- To investigate the impact of adding tactile cues to auditory-visual speech perception.
- To assess speech recognition in various sensory modalities.
Main Methods:
- Twenty adults performed speech recognition tasks in auditory-only, auditory-visual, auditory-tactile, and auditory-visual-tactile conditions.
- Stimuli were synchronized with target speech envelope at varying signal-to-noise ratios (SNRs).
Main Results:
- Multimodal cues (visual, tactile, or both) significantly improved word recognition, particularly at lower SNRs (-7 to -3 dB).
- The auditory-visual-tactile condition showed the greatest improvement (14-25%).
- Visual and tactile stimuli demonstrated additive effects on speech recognition.
Conclusions:
- Temporally synchronized multimodal cues enhance speech perception in noisy environments.
- Multisensory integration relies on salient temporal cues for improved speech recognition.
Related Concept Videos
01:21Perceiving Loudness, Pitch, and Location
395
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
395
01:17Auditory Perception
426
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
426
01:01Perception of Sound Waves
4.6K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
4.6K
01:22Chunking and Rehearsal in Sensory Memory
271
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
271
01:31Hearing
52.9K
When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.
52.9K
01:15Sensory Modalities
1.5K
Sensation typically is the process by which the sensory receptors and sense organs detect stimuli from the internal and external environment and transmit this information to the central nervous system for processing.
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
1.5K