Transformer based sign-to-text translation for Bangladeshi sign language

Area of Science:

• Computer Science
• Artificial Intelligence
• Linguistics

Background:

• Bangladeshi Sign Language (BdSL) is crucial for Deaf communities in Bangladesh.
• Existing BdSL translation tools lack real-time, word-level accuracy, hindering communication with non-signers.
• A practical, accurate, and efficient sign-to-text pipeline is needed.

Purpose of the Study:

• To develop a robust sign-to-text pipeline for 102 everyday BdSL words.
• To design a compact Transformer model for real-time BdSL translation.
• To benchmark the model's performance against existing methods and ensure reproducibility.

Main Methods:

• A feature-first pipeline converting video to MediaPipe hand and body landmarks.
• A lightweight Transformer encoder processing 30-frame sequences of skeletal data.
• Standardized data curation, preprocessing, and a clear training/evaluation recipe.

Main Results:

• The proposed model achieved 98.1% test accuracy.
• Outperformed stacked LSTM and SSD MobileNet V2 baselines.
• Demonstrated low latency, suitable for real-time application and on-device use.

Conclusions:

• The developed BdSL sign-to-text system is accurate, efficient, and privacy-friendly.
• The landmark-based approach is suitable for assistive communication in various settings.
• The system is extensible for incorporating additional BdSL vocabulary.