FluentSigners-50: A signer independent benchmark dataset for sign language processing

Area of Science:

• Computational Linguistics
• Computer Vision
• Human-Computer Interaction

Background:

• Sign Language Processing (SLP) research requires large-scale, diverse datasets for robust model development.
• Existing datasets often lack the variability needed to represent real-world sign language usage.
• Kazakh-Russian Sign Language (KRSL) has not been adequately represented in large-scale digital resources for SLP.

Purpose of the Study:

• Introduce the FluentSigners-50 dataset, a novel, large-scale, signer-independent resource for KRSL.
• Establish a new benchmark for evaluating Continuous Sign Language Recognition (CSLR) and Continuous Sign Language Translation (CSLT) systems.
• Facilitate research into more accurate and adaptable sign language recognition models.

Main Methods:

• Collected 43,250 video samples of 173 KRSL sentences from 50 signers.
• Recorded videos in diverse real-life conditions, including varied backgrounds, devices (smartphones, webcams), and camera parameters.
• Incorporated linguistic and inter-signer variability to simulate natural sign language use.
• Established baseline performance using Stochastic CSLR and TSPNet, state-of-the-art SLP methods.
• Created three distinct train-test splits focusing on signer, age, and unseen sentence independence.

Main Results:

• The FluentSigners-50 dataset comprises 43,250 video samples from 50 signers performing 173 sentences.
• The dataset captures significant linguistic and inter-signer variability, reflecting real-world KRSL communication.
• Baseline evaluations using Stochastic CSLR and TSPNet provide initial performance metrics on the dataset.
• The prepared train-test splits enable rigorous evaluation of model generalization capabilities.

Conclusions:

• The FluentSigners-50 dataset represents a significant advancement for KRSL research and sign language processing.
• The dataset's diversity and scale are crucial for developing robust CSLR and CSLT models.
• Public availability of FluentSigners-50 will foster further research and development in sign language technologies.