具有多正对比学习的生成手语描述提示符,用于手语识别
Siyu Liang1,2,3, Yunan Li1,2,3, Wentian Xin4
1School of Computer Science and Technology, Xidian University, Xi'an 710071, China.
Sensors (Basel, Switzerland)
|October 16, 2025
概括
本研究介绍了GSP-MC,这是一种用于标语识别的新方法,它使用生成的大型语言模型来准确地描述标语. 它实现了最先进的准确性,增强了包容性通信技术.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 语言学的语言学.
背景情况:
- 标语识别系统在等级语义上的多模式注释方面遇到了困难.
- 目前的方法缺乏捕捉顺序的手动和并发的非手动线索的能力.
研究的目的:
- 提出GSP-MC,这是第一个将生成大型语言模型 (LLM) 集成到手语识别中的方法.
- 开发一个能够精确地描述多部分和对手语的层次性语义理解的系统.
主要方法:
- 使用提取增强生成与特定领域的LLM和专家验证的公司.
- 采用双编码器架构,通过概率匹配将骨架特征与多层次文本描述 (全局,同义词,部分) 双向对齐.
- 综合全球和部分级损失与KL差异优化,以实现强大的对齐和语义捕获.
主要成果:
- 实现了最先进的精度:97.1%的中国SLR500和97.07%的土耳其AUTSL.
- 与现有方法 (SSRL和SML) 相比,证明了更高的性能.
- 证实了GSP-MC方法的跨语言潜力.
结论:
- GSP-MC有效地弥合了手语识别多式联络注释的差距.
- 该方法准确地捕捉了符号语义和详细的动态,从而实现了可靠的识别.
- 这项工作通过改善手语理解来推进包容性沟通技术.
相关概念视频
Sign Test for Matched Pairs
380
The sign test for matched pairs offers a robust method for comparing two paired samples, often for the effects of an intervention in one of them. This method is very useful in situations where the underlying distribution of the data is unknown. The test compares two related samples—often pre- and post-treatment measurements on the same subjects—to determine if there are significant differences in their median values.
To conduct the sign test, we first calculate the differences in...
To conduct the sign test, we first calculate the differences in...
380
Introduction to the Sign Test
1.3K
The sign test is an important tool in nonparametric statistics, offering a straightforward yet effective method for analyzing matched pairs, nominal data, or hypotheses concerning the median of a population. It transforms data points into positive or negative signs, avoiding the need for assumptions about data distribution and instead focusing on the direction of change. It is particularly valuable when data does not conform to the normal distribution requirements of many parametric tests. For...
1.3K
Learning Disabilities
570
Learning disabilities are cognitive disorders caused by neurological impairments that affect cognitive functions like language and reading, without indicating overall intellectual or developmental challenges. These disabilities differ from global intellectual or developmental disabilities as they are limited to distinct cognitive functions. Common learning disabilities include dysgraphia, dyslexia, and dyscalculia, each of which impacts unique aspects of learning.
Dyslexia
Dyslexia is a...
Dyslexia
Dyslexia is a...
570
Sign Test for Nominal Data
375
The sign test is a nonparametric method used to evaluate hypotheses about the median of a single sample or to compare the medians of two related samples. The sign test is particularly useful when dealing with nominal data, which includes distinct categories without an inherent order, such as names, labels, and preferences. Nominal data restricts statistical analysis to evaluating population proportions rather than mean or median values that require continuous data.
For example, consider a...
For example, consider a...
375
Sign Convention
3.4K
When analyzing a beam subjected to various loads, it is crucial to understand the internal forces and moments generated within the structure. These internal forces can be broadly classified into normal forces, shear forces, and bending moments. To determine these forces and moments, we use the method of sections and apply a specific sign convention based on their direction and the side of the section being analyzed.
The normal force acts perpendicular to the beam's cross-section and can...
The normal force acts perpendicular to the beam's cross-section and can...
3.4K
Purposive Learning
438
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
438


