相关实验视频

Updated: Jan 15, 2026

Stimulating the Lip Motor Cortex with Transcranial Magnetic Stimulation

Stimulating the Lip Motor Cortex with Transcranial Magnetic Stimulation

Published on: June 14, 2014

自然L2S:端到端高质量的多扬声器口腔到语音合成与差异数字信号处理

Yifan Liang¹, Fangkun Liu¹, Andong Li¹

¹Institute of Acoustics, Chinese Academy of Sciences, Beijing, 100190, China; University of Chinese Academy of Sciences, Beijing, 100049, China.

Neural networks : the official journal of the International Neural Network Society

|October 14, 2025

概括

本研究介绍了NaturalLip-to-Speech (NaturalL2S),这是一个端到端的框架,通过弥合Mel谱的域差距来改善嘴唇到语音合成. 自然L2S通过声学诱导先验和基本频率预测器来增强合成语音质量.

科学领域:

人工智能的人工智能
语音技术语言技术
计算机视觉计算机视觉

背景情况:

视觉语音识别 (VSR) 模型通过提供语义信息来改善口唇语音合成.
使用VSR模型的级联框架显示出希望,但在mel谱图中存在域间隙,降低了合成质量.

研究的目的:

提出自然的嘴唇到语音 (NaturalL2S),这是一个端到端的框架,以弥合Mel-spectrogram领域的差距.
从唇部运动来增强合成语音的可理解性和质量.

主要方法:

开发了一个端到端的框架 (NaturalL2S),共同培训具有声学感应先验的语音编码器.
引入了一个基本频率 (F0) 预测器来模拟前变异.
采用了可分辨数字信号处理 (DDSP) 合成器,由预测F0驱动,用于声学前一代.

主要成果:

在没有明确的扬声器嵌入的情况下实现了令人满意的扬声器相似性.
与基于客观指标和主观听力测试的最先进方法相比,合成语音质量显著提高.
有效地弥合了合成和真实金属光谱之间的领域差距.

结论:

通过整合声学先验和F0预测,NaturalL2S提供了一种新的口腔到语音合成方法.
该框架在语音合成质量和演讲者相似性方面表现出卓越的表现.
这种方法解决了当前VSR增强语音合成系统中的一个关键瓶.

关键词:

可差异化的数字信号处理过程.终端到终端的培训.嘴唇对口语的交换方式语音重建语言重建

更多相关视频

Synthetic, Multi-Layer, Self-Oscillating Vocal Fold Model Fabrication

Synthetic, Multi-Layer, Self-Oscillating Vocal Fold Model Fabrication

Published on: December 2, 2011

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

Published on: August 9, 2024

相关概念视频

Design Example

Design Example

The innovation of touch-tone telephony revolutionized the telecommunications industry by replacing the traditional rotary dial with a dual-tone multi-frequency (DTMF) signaling system. This system uses a matrix-style keypad with buttons arranged in four rows and three columns, creating 12 distinct signals each assigned to a pair of frequencies. Each button press results in a simultaneous generation of two sinusoidal tones – one from a low-frequency group (697 to 941 Hz) and one from a...

Integration by Parts: Problem Solving

Integration by Parts: Problem Solving

Smart speakers process voice commands by modeling audio inputs as piecewise functions and analyzing them through integration against trigonometric functions, such as cosine. This mathematical approach is fundamental in signal processing, where complex sound waves are decomposed into simpler frequency components.Consider a definite integral involving a piecewise function multiplied by a cosine function. Because the function is defined differently over separate intervals, the integral is split...