Related Experiment Video
Updated: Jun 20, 2026
05:48
Memorization-Based Training and Testing Paradigm for Robust Vocal Identity Recognition in Expressive Speech Using Event-Related Potentials Analysis
Published on: August 9, 2024
Deviance detection in congruent audiovisual speech: evidence for implicit integrated audiovisual memory
István Winkler1, János Horváth, Júlia Weisz
1Institute for Psychology, Hungarian Academy of Sciences, Hungary. iwinkler@cogpsyphy.hu
Biological Psychology
|September 8, 2009
Summary
Detecting deviant speech in noise improves with visual cues. An audiovisual mismatch negativity (avMMN) response suggests integrated cross-modal memory representations form even outside attention.
Area of Science:
- Neuroscience
- Cognitive Science
- Auditory Perception
Background:
- Investigating auditory perception in noisy environments is crucial for understanding speech processing.
- The oddball paradigm is a standard method for studying neural responses to infrequent stimuli.
- Multisensory integration, particularly audiovisual speech, enhances perception and neural processing.
Purpose of the Study:
- To examine the detection of deviant speech syllables in noise using an oddball paradigm.
- To investigate the role of visual speech in improving auditory detection.
- To characterize the event-related potential (ERP) correlates of audiovisual deviant speech detection.
Main Methods:
- Utilized an oddball paradigm with auditory speech syllables embedded in continuous noise.
- Included conditions with auditory-only, visual-only, and congruent audiovisual stimuli.
- Recorded event-related potentials (ERPs) to analyze neural responses to deviant stimuli.
Main Results:
- Behavioral performance improved in detecting and identifying speech syllables when visual speech was congruent.
- A unique negative ERP difference wave (peaking at ~290ms) was observed for audiovisual deviants, termed audiovisual mismatch negativity (avMMN).
- This avMMN showed a different scalp distribution compared to unimodal mismatch negativity (MMN) and visual MMN (vMMN).
Conclusions:
- The elicitation of avMMN indicates that integrated audiovisual memory representations are involved in detecting deviant speech.
- Formation of these cross-modal memory representations does not necessitate voluntary attention.
- These findings suggest that the brain automatically integrates multisensory information for speech perception, even for unattended stimuli.
More Related Videos
09:09Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
Published on: September 27, 2024
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
Related Concept Videos
01:22Eyewitness Memory
Eyewitness memory refers to the recollection of events by someone who has directly witnessed them, often serving as critical evidence in legal settings. This type of memory is commonly used in criminal cases where a witness describes details like a suspect's appearance, clothing, or behavior during a crime. However, despite its perceived reliability, eyewitness memory is prone to significant errors.
One such error is memory distortion, which occurs because human memory does not function like a...
One such error is memory distortion, which occurs because human memory does not function like a...
01:22Chunking and Rehearsal in Sensory Memory
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of information more...
01:29Non-Verbal Cues
Non-verbal communication extends beyond gestures and facial expressions to include vocal elements known as paralanguage. Paralanguage consists of non-verbal vocal cues such as pitch, loudness, speech rate, pauses, and non-verbal vocalizations like laughter, sighs, and moans. These elements not only accompany speech but also provide critical emotional and contextual information.The Role of Paralanguage in CommunicationParalanguage adds depth to spoken language by conveying emotions and...
01:17Auditory Perception
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the cochlea, a...
01:24Implicit Memories
Implicit memories, also known as non-declarative memories, are long-term memories that function outside of conscious awareness. These memories influence behavior and skills without explicit knowledge. This type of memory is evident in tasks like playing tennis, snowboarding, and texting. Implicit memory has three subsystems: procedural memory, conditioning, and priming. This type of memory is essential in various activities, from everyday tasks to specialized skills.
One key aspect of implicit...
One key aspect of implicit...
01:12Visual Agnosia
Visual agnosia is a condition characterized by the inability to recognize visually presented objects despite having normal vision. For instance, a person with visual agnosia can describe the shape and color of an object but cannot identify or name it. This impairment does not affect their visual field, acuity, color vision, brightness discrimination, language, or memory. An example of this condition in a social setting is someone at a dinner party asking for "that silver thing with a round end"...