一个可解释的混合CNN-变压器模型,用于边缘设备上的手语识别,使用自适应融合和知识蒸
Ismail Lamaakal1, Chaymae Yahyati2, Yassine Maleh3
1Multidisciplinary Faculty of Nador, Mohammed Premier University, Oujda, Morocco. ismail.lamaakal@ieee.org.
Scientific reports
|February 3, 2026
概括
TinyMSLR是一个新的手语识别 (SLR) 框架,它是高效,易于解释和轻量级的. 它在多语言数据集上实现了高精度,使其适合在资源有限的设备上实时通信.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 人与计算机的交互
背景情况:
- 用于手语识别 (SLR) 的深度学习 (DL) 面临着单语言数据,可解释性和边缘部署的计算强度的挑战.
- 人们越来越需要包容性,实时的通信技术,这凸显了高效和可部署的SLR系统的重要性.
研究的目的:
- 介绍TinyMSLR,一种可解释的,轻量级的框架,用于在资源有限的设备上进行隔离符号分类.
- 通过一种新的双教师知识蒸 (KD) 方案,提高多语言SLR的准确性和效率.
主要方法:
- TinyMSLR将ConvNeXt-Tiny和Swin变压器编码器与自适应融合门相结合.
- 双教师知识蒸 (KD) 方案将知识从大型模型转移到紧的学生模型.
- 对多语言数据集 (DGS-2014T,CSL) 进行了评估,重点是孤立标志识别.
主要成果:
- TinyMSLR实现了高精度:99.28%的训练,99.01%的验证,98.96%的F1分数.
- 这款车型重量轻,参数不到2.7M.
- 推理延迟在CPU上为24ms,在边缘GPU上不到13.5ms.
结论:
- TinyMSLR为多语言孤立标志识别提供了准确性,效率和可解释性的实用平衡.
- 该框架非常适合在边缘设备上部署,从而实现实时通信.
- 这项工作促进了对聋人和听力障碍者社区的包容性通信技术的发展.
更多相关视频
06:16Involving Individuals with Developmental Language Disorder and Their Parents/Carers in Research Priority Setting
Published on: June 6, 2020
4.6K
05:15The Spatial Memory Game: Testing the Relationship Between Spatial Language, Object Knowledge, and Spatial Cognition
Published on: February 19, 2018
11.4K
相关概念视频
Hybrid Zones
21.9K
Hybrid zones are narrow regions where two closely related species interact, mate, and produce hybrids. Relative to either parent species, hybrids may possess distinct phenotypic or genetic differences that impact their survival and reproductive success. The genetic variances introduced by hybridization influence species diversity and speciation processes within the hybrid zone.
21.9K
Hybridization of Atomic Orbitals I
67.5K
The mathematical expression known as the wave function, ψ, contains information about each orbital and the wavelike properties of electrons in an isolated atom. When atoms are bound together in a molecule, the wave functions combine to produce new mathematical descriptions that have different shapes. This process of combining the wave functions for atomic orbitals is called hybridization and is mathematically accomplished by the linear combination of atomic orbitals. The new orbitals that...
67.5K
Nuclear Fusion
33.9K
The process of converting very light nuclei into heavier nuclei is also accompanied by the conversion of mass into large amounts of energy, a process called fusion. The principal source of energy in the sun is a net fusion reaction in which four hydrogen nuclei fuse and ultimately produce one helium nucleus and two positrons.
A helium nucleus has a mass that is 0.7% less than that of four hydrogen nuclei; this lost mass is converted into energy during the fusion. This reaction produces about...
A helium nucleus has a mass that is 0.7% less than that of four hydrogen nuclei; this lost mass is converted into energy during the fusion. This reaction produces about...
33.9K
Language
917
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
917
Hybridization of Atomic Orbitals II
49.1K
sp3d and sp3d 2 Hybridization
49.1K
Sign Convention
3.5K
When analyzing a beam subjected to various loads, it is crucial to understand the internal forces and moments generated within the structure. These internal forces can be broadly classified into normal forces, shear forces, and bending moments. To determine these forces and moments, we use the method of sections and apply a specific sign convention based on their direction and the side of the section being analyzed.
The normal force acts perpendicular to the beam's cross-section and can...
The normal force acts perpendicular to the beam's cross-section and can...
3.5K
