Related Experiment Video
Updated: Jul 30, 2025
05:48
Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
Published on: August 9, 2024
1.6K
Explicit-memory multiresolution adaptive framework for speech and music separation
Ashwin Bellur1, Karan Thakkar1, Mounya Elhilali1
1Electrical and Computer Engineering, Johns Hopkins University, Baltimore, USA.
EURASIP Journal on Audio, Speech, and Music Processing
|May 14, 2023
Summary
This study introduces a unified computational framework for sound source separation, mimicking the human auditory system's use of memory and feedback. The model effectively separates speech and music, demonstrating domain-agnostic principles for enhanced auditory perception.
Area of Science:
- Auditory Neuroscience
- Computational Acoustics
- Signal Processing
Background:
- The human auditory system separates sound streams using multi-scale representations, memory, and feedback mechanisms.
- Existing sound source separation methods often treat speech and music domains separately.
Purpose of the Study:
- To propose a unified computational framework for sound source separation inspired by human auditory principles.
- To demonstrate domain-agnostic applicability of sound separation techniques for both speech and music.
Main Methods:
- Developed an end-to-end computational framework using parallel and hierarchical convolutional paths.
- Implemented explicit memory and self-feedback mechanisms to refine sound stream selection.
- Utilized temporal coherence for gating target stream embeddings.
Main Results:
- Achieved stable sound source separation for both speech and music mixtures.
- Demonstrated the effectiveness of explicit memory in guiding information selection from complex auditory inputs.
- Showcased the benefits of feedback mechanisms in improving sound selectivity.
Conclusions:
- A unified computational framework can effectively perform sound source separation across different domains (speech, music).
- Explicit memory and feedback are crucial for enhancing auditory selectivity in complex sound environments.
- The proposed model offers a domain-agnostic approach to sound source separation, mimicking biological auditory processing.
Related Concept Videos
01:20Downsampling
202
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
202
01:12Double Resonance Techniques: Overview
252
Double resonance techniques in Nuclear Magnetic Resonance (NMR) spectroscopy involve the simultaneous application of two different frequencies or radiofrequency pulses to manipulate and observe two distinct nuclear spins. One important application of double resonance is spin decoupling, which selectively suppresses coupling with one type of nucleus while observing the NMR signal from another nucleus, simplifying the spectrum and enhancing resolution.
Spin decoupling is usually achieved by...
Spin decoupling is usually achieved by...
252
01:10Reconstruction of Signal using Interpolation
267
Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next...
267
01:22Chunking and Rehearsal in Sensory Memory
256
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
256
01:21Perceiving Loudness, Pitch, and Location
283
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
283
01:07Elaborative Rehearsals
111
Elaborative rehearsal is a crucial cognitive strategy that strengthens information encoding in long-term memory by making meaningful connections between new data and pre-existing knowledge. This approach contrasts with maintenance rehearsal, which involves simple repetition without delving into the significance of the information. While maintenance rehearsal might temporarily keep information active in short-term memory, it is less effective for long-term retention.
The effectiveness of...
The effectiveness of...
111