Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Perceiving Loudness, Pitch, and Location01:21

Perceiving Loudness, Pitch, and Location

202
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
202
Hearing01:31

Hearing

51.9K
When we hear a sound, our nervous system is detecting sound waves—pressure waves of mechanical energy traveling through a medium. The frequency of the wave is perceived as pitch, while the amplitude is perceived as loudness.
51.9K
Auditory Pathway01:15

Auditory Pathway

5.3K
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
5.3K
Auditory Perception01:17

Auditory Perception

324
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
324
Parallel Processing01:20

Parallel Processing

146
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
146
Perception01:28

Perception

442
Perception is a fundamental psychological process that enables individuals to organize, interpret, and consciously experience sensory information. This process is crucial for understanding and interacting with the world around us. It includes both bottom-up and top-down processing, each playing a distinct role in how we perceive our environment.
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...
442

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Computed Fluid Dynamics-Based Blood Pressure Prediction for Coronary Artery Disease Diagnosis Using Coronary Computed Tomography Angiography.

Journal of imaging·2026
Same author

Enhancing Explainable AI Stability with Realistic Synthetic Data for Cardiovascular Risk Prediction.

Studies in health technology and informatics·2026
Same author

Ultra-high-field fMRI study of insular discrimination of taste quality and perceived taste valence.

Chemical senses·2026
Same author

Knowledge graph embedding and alignment of incomplete electronic health records for critical care applications.

Journal of biomedical semantics·2026
Same author

Device-Related Adverse Events and Outcomes in Patients With Temporary Mechanical Circulatory Support Placed at Referral Centers Versus Cardiogenic Shock Hub Centers: An Observational Analysis.

Circulation. Heart failure·2026
Same author

Pure global amusia in a professional opera singer.

Neurological sciences : official journal of the Italian Neurological Society and of the Italian Society of Clinical Neurophysiology·2026

Related Experiment Video

Updated: Jun 13, 2025

Interaction between Phonological and Semantic Processes in Visual Word Recognition using Electrophysiology
05:38

Interaction between Phonological and Semantic Processes in Visual Word Recognition using Electrophysiology

Published on: June 29, 2021

2.3K

Bridging auditory perception and natural language processing with semantically informed deep neural networks.

Michele Esposito1, Giancarlo Valente2, Yenisel Plasencia-Calaña3

  • 1Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, The Netherlands. m.esposito@maastrichtuniversity.nl.

Scientific Reports
|September 9, 2024
PubMed
Summary

This study shows that deep neural networks (DNNs) can better recognize sounds by using semantic information, similar to how humans do. This approach improves artificial hearing systems by capturing relationships between sounds.

Keywords:
Acoustic-to-semantic transformationAuditory perceptionCognitive neuroscienceDeep neural networksNatural language processingSemantic embeddingsSound recognition

More Related Videos

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications
03:31

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Published on: December 15, 2023

487
Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
11:39

Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique

Published on: September 7, 2022

2.1K

Related Experiment Videos

Last Updated: Jun 13, 2025

Interaction between Phonological and Semantic Processes in Visual Word Recognition using Electrophysiology
05:38

Interaction between Phonological and Semantic Processes in Visual Word Recognition using Electrophysiology

Published on: June 29, 2021

2.3K
Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications
03:31

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Published on: December 15, 2023

487
Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
11:39

Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique

Published on: September 7, 2022

2.1K

Area of Science:

  • Artificial Intelligence
  • Cognitive Neuroscience
  • Machine Learning

Background:

  • Human sound recognition effortlessly utilizes semantic information, a capability lacking in current artificial hearing systems.
  • Deep neural networks (DNNs), particularly CNNs, excel at sound classification but typically use categorical labels, ignoring semantic relationships.
  • Cognitive neuroscience suggests humans leverage semantic information alongside acoustic cues for superior sound recognition.

Purpose of the Study:

  • To investigate if incorporating semantic information into DNNs enhances sound recognition performance.
  • To develop a novel approach for sound recognition that better emulates human auditory perception.
  • To bridge the gap between artificial sound recognition systems and human auditory capabilities.

Main Methods:

  • Framed sound recognition as a regression problem, training CNNs to map spectrograms to continuous semantic representations.
  • Utilized Natural Language Processing (NLP) models like Word2Vec, BERT, and CLAP text encoder for semantic embeddings.
  • Trained two DNN types: semDNN (semantic continuous embeddings) and catDNN (categorical labels) on a large dataset of 388,211 sounds.

Main Results:

  • SemDNN demonstrated superior performance over catDNN across four external datasets, preserving higher-level semantic relations.
  • SemDNN better approximated human listener behavior in similarity ratings for natural sounds compared to catDNN and other models.
  • The study confirmed the hypothesis that semantic information integration significantly improves DNN sound recognition.

Conclusions:

  • Incorporating semantic information into DNNs is crucial for advancing artificial hearing systems.
  • The semDNN approach offers a more human-like sound recognition capability, moving beyond simple categorical classification.
  • This research highlights the importance of semantic understanding in developing more sophisticated AI for auditory perception.