Sign Language Motion Generation from Sign Characteristics.

Area of Science:

• Computer Science
• Artificial Intelligence
• Computational Linguistics

Background:

• Sign language motion generation is complex, requiring detailed representation of hand and body movements.
• Existing methods may lack the granularity needed for realistic sign language synthesis.
• Sign phonemes offer a structured way to encode essential sign characteristics.

Purpose of the Study:

• To propose and evaluate a novel deep learning architecture for generating sign language motion from sign phonemes.
• To develop and assess a stop detection module for accurate sign sequence termination.
• To optimize system performance through various strategies like padding, interpolation, and data augmentation.

Main Methods:

• Utilized a transformer-based deep learning architecture for sign motion generation.
• Employed HamNoSys (Hamburg Notation System) for detailed sign phoneme representation.
• Implemented dynamic time warping (DTW) for motion generation evaluation and ROC curves for stop detection accuracy.

Main Results:

• The best system configuration achieved an average DTW distance per frame of 0.1057 for motion generation.
• The stop detection module demonstrated an area under the ROC curve (AUC) greater than 0.94.
• Evaluated multiple strategies including padding, interpolation, and data augmentation to find optimal configurations.

Conclusions:

• The proposed transformer-based approach effectively generates detailed sign language motion from phonemes.
• The integrated stop detection module accurately predicts the end of sign sequences.
• The study provides a robust framework for automatic sign language motion synthesis.