Related Experiment Video
Updated: Aug 5, 2026
08:31
Three-dimensional Optical-resolution Photoacoustic Microscopy
Published on: May 3, 2011
Single-point optical-vibration sensing system for deep-learning-based stereo sound synthesis
Kuo-Wei Chao1, Ji-Yan Han1, Jia-Wei Chen1
1Department of Biomedical Engineering, National Yang Ming Chiao Tung University, Taiwan.
The Journal of the Acoustical Society of America
|August 3, 2026
Summary
This study introduces a novel deep learning system for stereo sound synthesis using laser Doppler vibrometry. The innovative method effectively reconstructs directional sound from vibration data, enhancing audio experiences.
Area of Science:
- Acoustics and Signal Processing
- Machine Learning for Audio Synthesis
- Vibrometry and Sensor Technology
Background:
- Traditional stereo recording using microphone arrays faces limitations in distance, noise, and setup complexity.
- Stereo sound significantly improves sound source localization and auditory immersion.
- Binaural hearing cues, such as interaural level difference (ILD) and interaural time difference (ITD), are crucial for stereo perception.
Purpose of the Study:
- To propose a novel deep-learning-based stereo sound synthesis system.
- To address the limitations of conventional stereo recording methods.
- To demonstrate the feasibility of synthesizing stereo sound from monophonic vibration signals.
Main Methods:
- Utilizing a laser Doppler vibrometer to capture monophonic vibration signals.
- Verifying the presence of encoded binaural hearing cues (ILD, ITD) within vibration signals.
- Employing a gated convolutional neural network for stereo sound synthesis from vibration data.
Main Results:
- Achieved high directional sound reconstruction rates (97.7%-98.8%) from single-point vibration data.
- Demonstrated scale-invariant signal-to-distortion ratio improvements of 2.64 (inside) and 3.52 (outside) in stereo music synthesis.
- Subjective evaluations showed significant increases in pleasantness (44.5%) and stereoscopic sense (57.0%).
Conclusions:
- The proposed deep learning system effectively synthesizes stereo sound from laser Doppler vibrometer signals.
- Monophonic vibration signals contain sufficient binaural cues for accurate stereo sound reconstruction.
- This approach offers a promising alternative to traditional stereo recording, enhancing audio quality and user experience.
Related Concept Videos
01:01Perception of Sound Waves
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same frequency...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same frequency...
01:21Perceiving Loudness, Pitch, and Location
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by identifying...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by identifying...