洛拉-INT8 Whisper:一个低成本的广东语语音识别框架,用于边缘设备
Lusheng Zhang1,2, Shie Wu1,2, Zhongxun Wang1,2
1School of Physics and Electronic Information, Yantai University, Yantai 264005, China.
Sensors (Basel, Switzerland)
|September 13, 2025
概括
本研究介绍了一种高效的广东语自动语音识别 (ASR) 系统,使用LoRA微调和INT8量化. 该方法显著减少了模型大小,并提高了低资源ASR应用的推断速度.
科学领域:
- 语音识别 语言识别
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 广东语的低资源自动语音识别 (ASR) 面临着数据稀缺,大型模型和缓慢推断的挑战.
- 边缘部署的ASR系统需要高效的模型,平衡准确性,速度和大小.
研究的目的:
- 为低资源和边缘部署设置开发一个具有成本效益的广东语ASR系统.
- 为了解决数据稀缺,模型大小和推断速度的瓶.
主要方法:
- 使用LoRA (低级调整) 对Whisper-tiny进行参数高效的微调,在Common Voice zh-HK数据集上排名=8.
- 在ONNX运行时使用动态量子化对微调模型进行INT8量子化,以创建一个紧的检查点.
- 对量子化模型在CPU和GPU上的性能进行评估,测量字符错误率 (CER) 和实时因子 (RTF).
主要成果:
- 微调LoRA将CER从49.5%降低到11.1%,接近完整的微调性能,同时显著降低了培训成本.
- INT8量子化模型实现了60MB的大小,CPU上的RTF=0.20 (5x实时) 和GPU上的RTF=0.06,显示了实质性的速度改进.
- 除研究证实LoRA-INT8配置在准确性,速度和模型大小之间提供了最佳的权衡.
结论:
- 拟议的LoRA-INT8广东语ASR系统为资源有限的环境提供了实际的解决方案.
- 该系统在效率和性能方面取得了显著的改进,为边缘设备的实时ASR铺平了道路.
- 未来的工作旨在进一步提高实际物联网应用的性能和能源效率.
更多相关视频
05:48Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
Published on: August 9, 2024
2.0K
06:22Machine Learning-Based Cough Tone Classification: Diagnostic Exploration of Chronic Obstructive Pulmonary Disease and Respiratory Tract Infections
Published on: September 19, 2025
435
相关概念视频
01:24Components of Language
752
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
752
01:21Perceiving Loudness, Pitch, and Location
942
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
942
01:21Larynx
4.2K
The human larynx, often referred to as the voice box, is an intricate organ located in the neck. It serves as a pathway for air to enter the lungs during respiration and is an essential component of voice production.
Anatomy of the Larynx
The larynx consists of various components, including cartilage, muscles, and vocal cords. Its structure includes three large unpaired cartilages—the thyroid, cricoid, and epiglottis—and three smaller paired cartilages—the arytenoids,...
Anatomy of the Larynx
The larynx consists of various components, including cartilage, muscles, and vocal cords. Its structure includes three large unpaired cartilages—the thyroid, cricoid, and epiglottis—and three smaller paired cartilages—the arytenoids,...
4.2K
01:22Language Development
853
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
853
01:20Downsampling
610
When considering a sampled sequence with zero values between sampling instants, one can replace it by taking every N-th value of the sequence. At these integer multiples of N, the original and sampled sequences coincide. This process, known as decimation, involves extracting every N-th sample from a sequence, thereby creating a more efficient sequence.
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
The Fourier transform of the decimated sequence reveals a combination of scaled and shifted versions of the original spectrum. This...
610
01:27Language and Cognition
716
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
716