一个非解剖学图形结构用于连续手语边界检测
Razieh Rastgoo1, Kourosh Kiani2, Sergio Escalera3
1Electrical and Computer Engineering Department, Semnan University, Semnan, 3513119111, Iran. rrastgoo@semnan.ac.ir.
Scientific reports
|July 15, 2025
概括
本研究介绍了一个结合图形卷积网络 (GCN) 和变压器模型的深度学习模型,用于在连续视频中精确检测手语边界. 该方法通过分析手关节运动和时间信息来增强标志识别.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 在连续的手语视频中检测孤立符号的边界是一个重大挑战.
- 现有的方法通常依赖于手工制作的功能,限制了性能和适应性.
- 整合手的结构和动态对于准确的标志识别至关重要.
研究的目的:
- 提出一种新的深度学习方法,用于在连续的手语视频中准确检测孤立符号的边界.
- 通过用GCN-Transformer架构替换手工制作的特征提取器来提高模型性能.
- 为了有效地利用手的结构和运动动态来改进标志边界检测.
主要方法:
- 一个两步的方法:预先培训孤立的标志视频和部署连续的标志视频.
- 利用图形卷积网络 (GCN) 进行丰富的空间特征提取和转换器模型进行时间信息处理.
- 引入一个非解剖学图形结构来表示手关节运动和关系,加上一个滑动窗口机制和一个后处理模块来检测最终的边界.
主要成果:
- 拟议的GCN-变压器模型在检测连续序列内的孤立信号边界方面表现出卓越的性能.
- 两个数据集的实验结果验证了该模型在解决手语边界检测复杂性的有效性.
- 非解剖手图结构和自我注意力机制对该模型的成功做出了重大贡献.
结论:
- 开发的深度学习方法有效地解决了在连续的手语视频中孤立的标志边界检测的挑战.
- 结合GCN,变压器和新的手图结构,为手语识别提供了一个强大的解决方案.
- 这项工作通过提供更准确和更适应的分析手语视频方法来推进该领域.
相关概念视频
01:18Signal Flow Graphs
319
Signal-flow graphs offer a streamlined and intuitive approach to representing control systems, providing an alternative to traditional block diagrams. These graphs use branches to symbolize systems and nodes to represent signals, effectively illustrating the relationships and interactions within the system.
In a signal-flow graph, branches denote the system's transfer functions, while nodes represent the signals. The direction of signal flow is indicated by arrows, with the corresponding...
In a signal-flow graph, branches denote the system's transfer functions, while nodes represent the signals. The direction of signal flow is indicated by arrows, with the corresponding...
319
01:07Bar Graph
19.6K
A bar graph is also called a bar chart and consists of bars that are separated from each other. It either uses horizontal or vertical bars to show comparisons among categories. The bars can be rectangles, or they can be rectangular boxes (used in three-dimensional plots). One axis of the graph represents the specific categories being compared, and the other axis shows a discrete value. In this graph, the length of the bar for each category is proportional to the number or percent of individuals...
19.6K
01:10Vector Algebra: Graphical Method
13.9K
Vectors can be multiplied by scalars, added to other vectors, or subtracted from other vectors. The vector sum of two (or more) vectors is called the resultant vector or, for short, the resultant.
We use the laws of geometry to construct resultant vectors, followed by trigonometry to find vector magnitudes and directions. For a geometric construction of the sum of two vectors in a plane, we follow the parallelogram rule. Suppose two vectors are at arbitrary positions. Translate either one of...
We use the laws of geometry to construct resultant vectors, followed by trigonometry to find vector magnitudes and directions. For a geometric construction of the sum of two vectors in a plane, we follow the parallelogram rule. Suppose two vectors are at arbitrary positions. Translate either one of...
13.9K
01:16SFG Algebra
175
In Signal Flow Graph (SFG) algebra, the value a node represents is determined by the sum of all signals entering that node. This summed value is then transmitted through every branch leaving the node, making the SFG a powerful tool for visualizing and analyzing control systems.
Each node in an SFG corresponds to a variable, and the interactions between nodes are represented by branches with associated gains. When multiple branches lead into a node, the value at that node is the sum of the...
Each node in an SFG corresponds to a variable, and the interactions between nodes are represented by branches with associated gains. When multiple branches lead into a node, the value at that node is the sum of the...
175
01:30Classification of Signals
903
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
903
01:20Structural Classification of Joints
4.2K
Joints, also known as articulations, are classified based on their structural characteristics, i.e., based on whether the articulating surfaces of the adjacent bones are directly connected by fibrous connective tissue or cartilage, or whether the articulating surfaces contact each other within a fluid-filled joint cavity. These differences serve to divide the joints of the body into three structural classifications.
A fibrous joint is where the adjacent bones are united by fibrous connective...
A fibrous joint is where the adjacent bones are united by fibrous connective...
4.2K