速度-扭曲-感知优化神经语音传输系统用于高保真语义通信
Shengshi Yao1, Zixuan Xiao1, Kai Niu1,2
1Key Laboratory of Universal Wireless Communications, Beijing University of Posts and Telecommunications, Beijing 100876, China.
Sensors (Basel, Switzerland)
|May 25, 2024
概括
这项研究介绍了神经语音传输 (NST),这是有效传输语音的新框架. 通过根据语音内容的复杂性适应地分配传输速率,NST可以提高速率-扭曲-感知性能.
科学领域:
- 语音处理和通信系统.
- 机器学习用于信号传输.
背景情况:
- 现有的语音传输联合源通道编码 (JSCC) 方法与跨的内容多样性作斗争,导致传输效率低下.
- 目前的方法缺乏基于语音内容复杂性的自适应性速率分配.
研究的目的:
- 提出一种新的神经语音传输 (NST) 框架,用于高保真语义通信.
- 通过解决现有语音传输方法的局限性来优化速度-扭曲-感知 (RDP) 性能.
主要方法:
- 开发了一个神经语音传输 (NST) 框架,使用学习的模型来评估语音特征的复杂性.
- 基于量化语义内容复杂性的实现适应性传输速率分配.
- 使用可变长度的联合源-通道编码来无集成源内容和通道状态信息.
- 引入了NST的流媒体变体,采用因果编码和滑动窗口来实现低延迟传输.
主要成果:
- 与现有的基于分离和JSCC方法相比,NST表现出优越的RDP性能.
- 学习模型有效量化语义内容复杂性,用于适应性速率分配.
- 流式NST实现了低延迟传输,质量降低最小,适合实时应用.
结论:
- 在学习语音传输方面,NST提供了显著的进步,优化了语义通信的RDP性能.
- 适应性速率分配和可变长度编码策略最大限度地提高了编码收益和传输效率.
- 流式NST为需要低延迟的实时语音通信提供了可行的解决方案.
相关概念视频
01:10Reconstruction of Signal using Interpolation
194
Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next...
194
01:22Upsampling
229
Managing signal sampling rates is essential in digital signal processing to maintain signal integrity. A decimated signal, characterized by a reduced frequency range due to its lower sampling rate, can be upsampled by inserting zeros between each sample. This upsampling process expands the original spectrum and introduces repeated spectral replicas at intervals dictated by the new Nyquist frequency. To refine this zero-inserted sequence, it is passed through a lowpass filter with a cutoff...
229
01:18Aliasing
130
Accurate signal sampling and reconstruction are crucial in various signal-processing applications. A time-domain signal's spectrum can be revealed using its Fourier transform. When this signal is sampled at a specific frequency, it results in multiple scaled replicas of the original spectrum in the frequency domain. The spacing of these replicas is determined by the sampling frequency.
If the sampling frequency is below the Nyquist rate, these replicas overlap, preventing the original...
If the sampling frequency is below the Nyquist rate, these replicas overlap, preventing the original...
130
01:20Downsampling
154
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
154
01:20¹³C NMR: Distortionless Enhancement by Polarization Transfer (DEPT)
1.0K
When proton-coupled carbon-13 spectra are simplified by a broadband proton decoupling technique, structural information about the coupled protons is lost. Distortionless enhancement by polarization transfer (DEPT) is a technique that provides information on the number of hydrogens attached to each carbon in a molecule. While the DEPT experiment utilizes complex pulse sequences, the pulse delay and flip angle are specifically manipulated. The resulting signals have different phases depending on...
1.0K
01:23Lossless Lines
123
In electrical engineering, a lossless transmission line is characterized by a purely imaginary propagation constant and a resistive characteristic impedance. The ABCD parameters, which describe the relationship between the input and output voltages and currents, indicate an equivalent π circuit with an imaginary series impedance and a shunt admittance. This results in a transmission line that, when the product of the phase constant (beta) and the length of the line is less than pi,...
123