Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Perception of Sound Waves01:01

Perception of Sound Waves

4.5K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
4.5K
Auditory Pathway01:15

Auditory Pathway

5.5K
Auditory pathways constitute the complex neural circuits responsible for transmitting and interpreting auditory information from the peripheral auditory system to the brain. Sound waves are initially captured by the outer ear, funneled through the ear canal, and reach the tympanic membrane (eardrum). These vibrations are transmitted via the middle ear's ossicles to the inner ear's cochlea.
When viewed cross-sectionally, the cochlea reveals the scala vestibuli and scala tympani flanking...
5.5K
Auditory Perception01:17

Auditory Perception

384
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
384
Classification of Signals01:30

Classification of Signals

543
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
543
Perceiving Loudness, Pitch, and Location01:21

Perceiving Loudness, Pitch, and Location

241
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
241
Force Classification01:22

Force Classification

1.3K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.3K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Mechanisms of Reversible Transition in Emulsions Stabilized by Modified Nanocrystalline Cellulose.

Molecules (Basel, Switzerland)·2026
Same author

A Calcium Silicate-derived Silicate Ion Solution Improves Fat Graft Retention and Promotes Angiogenesis: An Experimental Study.

Aesthetic surgery journal·2026
Same author

SAGD: Boundary-Enhanced Segment Anything in 3D Gaussian via Gaussian Decomposition.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2026
Same author

Injectable natural hydrogel-loaded Ta<sub>4</sub>C<sub>3</sub> quantum dots self-assembled with Phillyrin-mediated NIR-II anti-infection for acute sinusitis.

Communications chemistry·2026
Same author

Loss of TRAIP Could Attenuate the Breast Cancer Cells Development by Regulating PLSCR4 Stabilization.

Journal of biochemical and molecular toxicology·2026
Same author

PMEPA1 modulates YAP1 nuclear translocation to disrupt EMT subtypes and promote metastasis in Biliary tract cancer.

Cell death & disease·2026
Same journal

Hidden Data Recovery and Forecasting via Next-Generation Reservoir Computing With Multiscale Delay Selection.

IEEE transactions on neural networks and learning systems·2026
Same journal

CAFF-CIL: Causality-Aware Freedom Forgetting Approach for Class-Incremental Learning.

IEEE transactions on neural networks and learning systems·2026
Same journal

Harmonic Autoencoding Framework for Multiple Tasks in Magnetic Particle Imaging Reconstruction.

IEEE transactions on neural networks and learning systems·2026
Same journal

A Survey on Human-Centric Voice-Face Multimodal Learning.

IEEE transactions on neural networks and learning systems·2026
Same journal

Vision-Assisted Foundation Model for Solving Multitask Vehicle Routing Problems.

IEEE transactions on neural networks and learning systems·2026
Same journal

FP3O: Enabling Proximal Policy Optimization in Multiagent Cooperation With Parameter-Sharing Versatility.

IEEE transactions on neural networks and learning systems·2026
See all related articles

Related Experiment Video

Updated: Jul 23, 2025

Cross-Modal Multivariate Pattern Analysis
13:51

Cross-Modal Multivariate Pattern Analysis

Published on: November 9, 2011

20.0K

Visually Guided Sound Source Separation With Audio-Visual Predictive Coding.

Zengjie Song, Zhaoxiang Zhang

    IEEE Transactions on Neural Networks and Learning Systems
    |July 12, 2023
    PubMed
    Summary
    This summary is machine-generated.

    This study introduces audio-visual predictive coding (AVPC), a parameter-efficient method for visually guided sound source separation. AVPC enhances performance by integrating audio and visual information recursively, outperforming existing models.

    More Related Videos

    Detecting Pre-Stimulus Source-Level Effects on Object Perception with Magnetoencephalography
    09:25

    Detecting Pre-Stimulus Source-Level Effects on Object Perception with Magnetoencephalography

    Published on: July 26, 2019

    6.9K
    Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder
    09:13

    Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder

    Published on: April 22, 2015

    16.6K

    Related Experiment Videos

    Last Updated: Jul 23, 2025

    Cross-Modal Multivariate Pattern Analysis
    13:51

    Cross-Modal Multivariate Pattern Analysis

    Published on: November 9, 2011

    20.0K
    Detecting Pre-Stimulus Source-Level Effects on Object Perception with Magnetoencephalography
    09:25

    Detecting Pre-Stimulus Source-Level Effects on Object Perception with Magnetoencephalography

    Published on: July 26, 2019

    6.9K
    Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder
    09:13

    Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder

    Published on: April 22, 2015

    16.6K

    Area of Science:

    • Artificial Intelligence
    • Computer Vision
    • Signal Processing

    Background:

    • Traditional visually guided sound source separation uses a divide-and-conquer approach.
    • This method is parameter-inefficient and challenging to optimize holistically.

    Purpose of the Study:

    • To present a novel, parameter-efficient, and effective approach for visually guided sound source separation.
    • To improve the integration of audio and visual information for better sound separation.

    Main Methods:

    • Developed audio-visual predictive coding (AVPC), a unified network architecture.
    • Utilized a ResNet-based network for visual features and a predictive coding network for audio processing and fusion.
    • Implemented a self-supervised learning strategy via co-predicting audio-visual representations.

    Main Results:

    • AVPC integrates audio and visual information recursively by minimizing prediction errors.
    • The proposed method significantly reduces model size compared to baselines.
    • AVPC demonstrates superior performance in separating musical instrument sounds.

    Conclusions:

    • AVPC offers a more effective and parameter-efficient alternative to existing methods.
    • The unified architecture simplifies the sound separation process.
    • Recursive integration of multimodal information leads to improved performance.