Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Perception of Sound Waves01:01

Perception of Sound Waves

The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same frequency...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Voice disorders classification using machine learning: a scoping review.

Frontiers in digital health·2026
Same author

Enhancing binaural speech perception in noise via weighted coherence masking for hearables.

The Journal of the Acoustical Society of America·2026
Same author

The Leaky Integrate-and-Fire Neuron Is a Change-Point Detector for Compound Poisson Processes.

Neural computation·2025
Same author

Low-latency hierarchical routing of reconfigurable neuromorphic systems.

Frontiers in neuroscience·2025
Same author

Spiking neural networks on FPGA: A survey of methodologies and recent advancements.

Neural networks : the official journal of the International Neural Network Society·2025
Same author

The neurobench framework for benchmarking neuromorphic computing algorithms and systems.

Nature communications·2025

Related Experiment Video

Updated: May 24, 2026

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
11:18

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks

Published on: March 2, 2015

Neuromorphic audio-visual sensor fusion on a sound-localizing robot.

Vincent Yue-Sek Chan1, Craig T Jin, André van Schaik

  • 1School of Electrical and Information Engineering, The University of Sydney Sydney, NSW, Australia.

Frontiers in Neuroscience
|February 21, 2012
PubMed
Summary

This study introduces a novel robotic system using neuromorphic audio-visual (AV) sensor fusion for sound localization. The robot achieved accurate sound source identification in reverberant conditions, demonstrating effective AV event binding.

Keywords:
neuromorphic engineeringonline learningsensor fusionsound localization

More Related Videos

Real-Time Proxy-Control of Re-Parameterized Peripheral Signals using a Close-Loop Interface
11:54

Real-Time Proxy-Control of Re-Parameterized Peripheral Signals using a Close-Loop Interface

Published on: May 8, 2021

Related Experiment Videos

Last Updated: May 24, 2026

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
11:18

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks

Published on: March 2, 2015

Real-Time Proxy-Control of Re-Parameterized Peripheral Signals using a Close-Loop Interface
11:54

Real-Time Proxy-Control of Re-Parameterized Peripheral Signals using a Close-Loop Interface

Published on: May 8, 2021

Area of Science:

  • Robotics
  • Artificial Intelligence
  • Sensor Fusion
  • Neuromorphic Engineering

Background:

  • Accurate sound localization is crucial for robotic perception and interaction.
  • Traditional audio-visual (AV) systems often lack the efficiency and adaptability of biological systems.
  • Neuromorphic sensors offer bio-inspired, low-power, and high-speed processing capabilities.

Purpose of the Study:

  • To develop and evaluate the first robotic system integrating audio-visual sensor fusion with neuromorphic sensors.
  • To enable a robot to learn sound localization using self-motion and visual feedback.
  • To investigate the audio-visual source binding problem using onset time matching.

Main Methods:

  • Integration of silicon cochleae (audio) and a silicon retina (visual) on a robotic platform.
  • Implementation of an adaptive Interaural Time Difference (ITD)-based sound localization algorithm.
  • Training the robot using self-motion and visual feedback for sound source learning.
  • Conducting experiments to assess AV source binding based on event onset synchrony.

Main Results:

  • The robotic system achieved an Root Mean Square (RMS) error of 4-5° in azimuth for sound source localization in reverberant environments.
  • The system demonstrated successful learning of sound localization through self-motion and visual cues.
  • The AV source binding experiment achieved a 75% correct match rate for audio-visual events, despite background noise and visual clutter.

Conclusions:

  • The developed neuromorphic AV sensor fusion system provides a robust platform for robotic sound localization.
  • The adaptive ITD-based algorithm effectively enables robots to learn and perform sound localization.
  • Onset time synchrony is a viable strategy for AV source binding in complex environments.