实时单眼基于骨的手势识别使用3D关节成型器.
Enmin Zhong1, Carlos R Del-Blanco1, Daniel Berjón1
1Grupo de Tratamiento de Imágenes (GTI), Information Processing and Telecommunications Center, ETSI Telecomunicación, Universidad Politécnica de Madrid, 28040 Madrid, Spain.
Sensors (Basel, Switzerland)
|August 26, 2023
概括
本研究介绍了一种混合3D-CNN和变压器模型,用于实时手势识别. 该方法通过在骨架数据中有效捕获本地和远程时间依赖性来实现高准确度.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 自动手势识别对于手语解释和家庭自动化等应用至关重要.
- 实时性能和管理时间依赖性是当前方法的关键挑战.
- 现有的3D卷积神经网络 (3D-CNNs) 和变压器模型在准确性和效率方面存在局限性.
研究的目的:
- 开发一种混合方法,将3D-CNN和变压器结合起来,用于增强手势识别.
- 提高实时识别能力,同时有效处理时间数据依赖性.
- 在准确性和处理速度方面超越现有的最先进的方法.
主要方法:
- 一个混合模型,集成3D-CNNs用于语义骨架嵌入和变压器用于长距离的时间依赖性捕获.
- 在变压器网络中利用自我注意机制.
- 在Briareo和多模式手势数据集上评估模型.
主要成果:
- 在Briareo数据集上获得了95.49%的高准确性得分,在多式手势数据集上达到97.25%的高准确性得分.
- 使用标准CPU,不需要GPU的实时识别性能得到证明.
- 在准确性和处理速度方面,超越了现有的方法.
结论:
- 混合3D-CNN和变压器模型为实时手势识别提供了卓越的解决方案.
- 这种方法有效地解决了局部和远程时间依赖的挑战.
- 该方法在该领域取得了重大进展,提供了高精度和计算效率.
相关概念视频
Structural Classification of Joints
3.5K
Joints, also known as articulations, are classified based on their structural characteristics, i.e., based on whether the articulating surfaces of the adjacent bones are directly connected by fibrous connective tissue or cartilage, or whether the articulating surfaces contact each other within a fluid-filled joint cavity. These differences serve to divide the joints of the body into three structural classifications.
A fibrous joint is where the adjacent bones are united by fibrous connective...
A fibrous joint is where the adjacent bones are united by fibrous connective...
3.5K
Functional Classification of Joints
4.2K
Functional Classification of Joints
The functional classification of joints is determined by the amount of mobility between the adjacent bones. Joints are functionally classified as a synarthrosis or immobile joint, an amphiarthrosis or slightly moveable joint, or as a diarthrosis, a freely moveable joint. Fibrous and cartilaginous joints can be functionally classified as either synarthroses or amphiarthroses, whereas all synovial joints are classified as diarthroses.
Synarthrosis
An...
The functional classification of joints is determined by the amount of mobility between the adjacent bones. Joints are functionally classified as a synarthrosis or immobile joint, an amphiarthrosis or slightly moveable joint, or as a diarthrosis, a freely moveable joint. Fibrous and cartilaginous joints can be functionally classified as either synarthroses or amphiarthroses, whereas all synovial joints are classified as diarthroses.
Synarthrosis
An...
4.2K
Muscles of the Forearm that Move the Hand and Fingers
1.0K
The muscles of the forearm that move the wrist, hand, and digits are numerous and diverse. They can be classified into two groups based on their location and function — the anterior and posterior compartment muscles.
Anterior Compartment
The anterior compartment muscles originate from the humerus. They primarily function as flexors and are also known as flexor muscles. They typically insert on the carpals, metacarpals, and phalanges. The superficial layer includes the flexor carpi...
Anterior Compartment
The anterior compartment muscles originate from the humerus. They primarily function as flexors and are also known as flexor muscles. They typically insert on the carpals, metacarpals, and phalanges. The superficial layer includes the flexor carpi...
1.0K
Muscles for Facial Expressions
2.2K
The craniofacial muscles are a collection of approximately 20 thin skeletal muscles situated beneath the skin of the face and scalp. These muscles, primarily responsible for the vast array of human facial expressions, originate from the bones or fibrous structures of the skull and extend outwards to connect with the skin. While most skeletal muscles in the body are enveloped in thick fascia, facial muscles generally have a more delicate fascial covering, with the buccinator muscle being a...
2.2K
Three-Dimensional Force System
2.1K
In mechanical engineering, a three-dimensional force system is a system of forces acting in three dimensions, with forces applied along the x, y, and z coordinate axes. The three-dimensional force system is an important concept in mechanical engineering, as it allows engineers to understand and analyze the behavior of objects and structures in three dimensions. By understanding the forces acting on a system, engineers can design more efficient and effective mechanical systems that can withstand...
2.1K
Bones of the Upper Limb: Humerus
3.3K
The upper limb consists of the arm, forearm, wrist, and hand bones. The humerus is the single bone of the upper arm region. Proximally, it has a large, spherical, smooth head that articulates with the glenoid cavity of the scapula to form the glenohumeral or shoulder joint. The margin of the head is the anatomical neck, a residual epiphyseal plate. Laterally it extends to form bony projections called the greater tubercle and the lesser tubercle. Next to the tubercles is the surgical neck, a...
3.3K


