Spatial Attention-Based 3D Graph Convolutional Neural Network for Sign Language Recognition

Area of Science:

• Computer Science
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Sign language is crucial for hearing-impaired individuals' communication.
• Mastering sign language is challenging for non-native users, creating a communication barrier.
• Sign language recognition (SLR) technology aims to overcome these challenges.

Purpose of the Study:

• To develop an efficient and effective architecture for sign language recognition.
• To address the complexities of manual and non-manual parameters in sign language.
• To bridge the communication gap between hearing-impaired and hearing individuals.

Main Methods:

• Proposed an architecture based on a convolutional graph neural network (GCN).
• Utilized a few separable 3D GCN layers for efficiency and to avoid over-smoothing.
• Incorporated a spatial attention mechanism to enhance gesture representation.

Main Results:

• The proposed GCN architecture demonstrated outstanding performance on various datasets.
• The spatial attention mechanism effectively improved the representation of spatial contexts in gestures.
• The limited-layer design mitigated the over-smoothing issue common in deep GCNs.

Conclusions:

• The developed GCN-based architecture is efficient and effective for sign language recognition.
• The integration of spatial attention significantly boosts recognition accuracy.
• This approach offers a promising solution for improving communication accessibility for the hearing-impaired community.