Related Experiment Video
Updated: Aug 13, 2026
07:14
Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models
Published on: December 23, 2025
LiteSwitchCodec: Neural speech coding with token space quantization and causal U-Net for personalized real-time
Xusheng Yang1, Wei Xiao2, Zixiang Wan1
1Advanced Data and Signal Processing Laboratory, School of Electronic and Computer Engineering, Peking University, Shenzhen, China.
Summary
LiteSwitchCodec offers high-quality speech compression and personalized real-time communication (PRTC) with enhanced security. This lightweight neural speech codec significantly reduces model parameters while maintaining excellent audio quality and low latency for PRTC services.
Area of Science:
- Speech Processing and Communication Systems
- Machine Learning for Audio Signal Processing
- Cybersecurity in Communication Networks
Background:
- Personalized real-time communication (PRTC) demands low latency, complexity, and robust security.
- Existing speech codecs may not adequately address the unique requirements of PRTC, including voice adaptation and copyright protection.
- The integration of voice adaptation (VA) into communication systems presents challenges in maintaining speech quality and security.
Purpose of the Study:
- To introduce LiteSwitchCodec, a novel lightweight neural speech codec designed for PRTC services.
- To enable high-quality speech compression and facilitate personalized voice adaptation within operator-managed platforms.
- To enhance security by protecting voice copyrights and preventing unauthorized impersonation in real-time communication.
Main Methods:
- Developed LiteSpeechCodec using fully causal convolutional layers for encoder/decoder with a mirrored structure for reduced decoder complexity.
- Implemented scalar quantization (SQ) as an alternative to residual vector quantization (RVQ) to simplify model architecture and improve compression.
- Introduced a lightweight causal U-Net model in the token space for personalized voice adaptation, trained using a two-stage strategy on public and custom voice conversion datasets.
Main Results:
- LiteSwitchCodec achieved a 38x reduction in model parameters compared to state-of-the-art codecs, maintaining real-time communication (RTC) quality.
- The codec demonstrated objective quality (ViSQOL: 4.32 at 7.2 kbps) and real-time voice adaptation with low latency (40 ms).
- Achieved superior performance in subjective and objective metrics for voice conversion in RTC, with 92.61% Resemblyzer similarity and a high speaker-similarity score (S-MOS: 4.67).
Conclusions:
- LiteSwitchCodec effectively balances high-quality speech compression, real-time voice adaptation, and low latency for PRTC applications.
- The codec inherently addresses security concerns, safeguarding voice copyrights and preventing impersonation, crucial for commercial RTC deployments.
- This approach offers a viable solution for next-generation communication systems prioritizing personalized experiences and robust security.
Related Concept Videos
01:28Neuronal Communication
Neurons, the fundamental units of the brain and nervous system, communicate through complex electrochemical signals that underpin all cognitive and bodily functions. This communication is primarily facilitated by a process involving the generation and propagation of an action potential along the axon of the neuron. When the internal electrical charge of a neuron surpasses a certain threshold, an action potential is triggered. This rapid change in voltage travels swiftly along the axon to the...
01:26Signal and System
A signal x(t) is a set of data or a time function representing a variable of interest. Signals typically convey information about a phenomenon, such as atmospheric temperature, humidity, human voice, television images, a dog's bark, or birdsongs. More generally, a signal can be a function of more than one independent variable. For instance, images depend on horizontal and vertical positions and can be regarded as two-dimensional signals. However, this text will focus on one-dimensional signals...
01:25Neural Circuits
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
01:27Language and Cognition
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
01:22Neurons as Communicators of the Brain
Neurons, the fundamental units of the brain and nervous system, function as the primary transmitters of information throughout the body. Their ability to communicate through electrical and chemical signals is vital for every bodily function, from regulating the heartbeat to processing complex thoughts. Each neuron has three main components: the cell body (soma), dendrites, and an axon, each specialized to facilitate swift and efficient neural communication.
Cell Body
The cell body, also known...
Cell Body
The cell body, also known...
01:10Higher Mental Functions of the Brain: Language
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...