使用对比的VideoMoCo框架进行自主监督学习,用于使用3D卷积网络识别沙特阿拉伯手语
Mahmoud Rokaya1, Dalia I Hemdan2, Mohammed A Alzain3
1Department of Information Technology, College of Computers and Information Technology, Taif University, 21944, Taif, Saudi Arabia. mahmoudrokaya@tu.edu.sa.
Scientific reports
|November 13, 2025
概括
本研究介绍了沙特阿拉伯手语 (SArSL) 认可的自我监督学习框架,达到92.7%的F1分数. 该方法通过改进的手势识别来提高沙特聋人社区的可访问性.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 人与计算机的交互
背景情况:
- 由于复杂的时空动态和有限的注释数据,沙特阿拉伯手语 (SArSL) 的识别具有挑战性.
- 现有的方法难以应对SArSL的复杂性,阻碍了沙特聋人社区的有效沟通.
研究的目的:
- 开发一个强大的和可扩展的自我监督的学习框架,以准确地识别沙特阿拉伯手语.
- 提高SArSL识别系统的性能和可访问性.
主要方法:
- 开发了一个使用视频动量对比 (VideoMoCo) 和3D ResNet-50骨干的自主监督学习框架.
- 该模型在18000个未标记的手势视频上进行了预训练,并在KARSL-502数据集 (15,400个样本,502个类) 上进行了微调.
主要成果:
- 拟议的框架实现了92.7%的F1得分,明显优于基线模型 (CNN-LSTM: 86.0%,双流CNN: 84.5%).
- 对类不平衡,运动变化和视觉噪音的强度已被证明,推理延迟为每批12ms的低推理延迟.
- 除研究证实了动量编码器和负样本队列在特征学习中的有效性.
结论:
- 视频MoCo-ResNet-50框架为实时SArSL识别提供了一个可扩展和包容的基础.
- 这一进步提高了沙特聋人社区的可访问性,并支持未来的多式联运应用.
相关概念视频
Force Classification
2.3K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
2.3K
Introduction to Learning
912
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
912
Observational Learning
804
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
804
Deconvolution
534
Deconvolution, also known as inverse filtering, is the process of extracting the impulse response from known input and output signals. This technique is vital in scenarios where the system's characteristics are unknown, and they must be inferred from the observable signals.
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
Deconvolution involves several mathematical techniques to derive the impulse response. One common approach is polynomial division. In this method, the input and output sequences are treated as coefficients of...
534
Learning Disabilities
559
Learning disabilities are cognitive disorders caused by neurological impairments that affect cognitive functions like language and reading, without indicating overall intellectual or developmental challenges. These disabilities differ from global intellectual or developmental disabilities as they are limited to distinct cognitive functions. Common learning disabilities include dysgraphia, dyslexia, and dyscalculia, each of which impacts unique aspects of learning.
Dyslexia
Dyslexia is a...
Dyslexia
Dyslexia is a...
559
Classification of Signals
1.3K
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
1.3K
