光前导向视觉特征学习,用于持续的手语识别
概括
本研究引入了光泽先导网络 (GPGN),通过提取可概括的视觉特征来改进连续手语识别 (CSLR). 通过利用光泽信息作为先验,GPGN增强了CSLR模型,提高了手语基准的性能.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 自然语言处理自然语言处理.
背景情况:
- 持续的手语识别 (CSLR) 旨在从视频数据中解释手语.
- 在CSLR中改善视觉特征提取器的泛化对于现实应用至关重要.
- 现有的方法经常在签名风格和环境条件的变化中扎.
研究的目的:
- 为了提高CSLR视觉特征提取器的概括能力.
- 引入一个新的光泽先导网络 (GPGN),利用光泽信息作为先导.
- 提高连续手语识别系统的准确性和稳定性.
主要方法:
- 使用预训练的光泽BERT模型来提取签名者不变的光泽特征.
- 建议建立一个光泽先导网络 (GPGN),并有一个并行密集连接的时间特征提取 (PDC-TFE) 模块.
- 交叉模式匹配,用Sinkhorn算法解决的规范化最佳运输问题来制定,指导视觉特征学习.
- 通过使用跨模式匹配损失和连接式时间分类 (CTC) 损失的组合来训练GPGN.
主要成果:
- 拟议的GPGN在德国和中国的手语识别基准上取得了竞争性表现.
- 废弃性研究证实了关键GPGN组件的有效性,包括PDC-TFE模块和跨模式匹配.
- 预训练的光泽BERT模型和跨模式匹配方法展示了用于增强现有的CSLR方法的插件和操作能力.
结论:
- 通过结合光泽先验,GPGN有效地提高了CSLR中视觉特征提取器的概括性.
- 拟议的方法在持续的手语识别准确性和稳定性方面取得了重大进展.
- 开发的技术可以很容易地集成到其他基于RGB的CSLR系统中,以提高其性能.
更多相关视频
07:12Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss
Published on: April 11, 2025
323
05:48Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
Published on: August 9, 2024
1.5K
相关概念视频
Introduction to the Sign Test
801
The sign test is an important tool in nonparametric statistics, offering a straightforward yet effective method for analyzing matched pairs, nominal data, or hypotheses concerning the median of a population. It transforms data points into positive or negative signs, avoiding the need for assumptions about data distribution and instead focusing on the direction of change. It is particularly valuable when data does not conform to the normal distribution requirements of many parametric tests. For...
801
Classification of Signals
441
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
441
Sign Test for Matched Pairs
124
The sign test for matched pairs offers a robust method for comparing two paired samples, often for the effects of an intervention in one of them. This method is very useful in situations where the underlying distribution of the data is unknown. The test compares two related samples—often pre- and post-treatment measurements on the same subjects—to determine if there are significant differences in their median values.
To conduct the sign test, we first calculate the differences in...
To conduct the sign test, we first calculate the differences in...
124
Sign Test for Nominal Data
91
The sign test is a nonparametric method used to evaluate hypotheses about the median of a single sample or to compare the medians of two related samples. The sign test is particularly useful when dealing with nominal data, which includes distinct categories without an inherent order, such as names, labels, and preferences. Nominal data restricts statistical analysis to evaluating population proportions rather than mean or median values that require continuous data.
For example, consider a...
For example, consider a...
91
Force Classification
1.2K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.2K
Associative Learning
335
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
335
