A hybrid CNN-transformer framework optimized by Grey Wolf Algorithm for accurate sign language recognition

Area of Science:

• Computer Vision
• Artificial Intelligence
• Deep Learning

Background:

• Accurate recognition of dynamic hand gestures, particularly American Sign Language (ASL), is crucial for assistive communication.
• Existing models face challenges in effectively capturing both spatial and temporal features for complex sign language recognition.

Purpose of the Study:

• To introduce and evaluate the Gray Wolf Optimized Convolutional Transformer Network (GWO-CTransNet) for enhanced dynamic hand gesture recognition.
• To leverage a hybrid deep learning approach combining CNNs, Transformers, and GWO for superior performance.

Main Methods:

• Developed a hybrid deep learning framework integrating Convolutional Neural Networks (CNNs) for spatial feature extraction and Transformers for temporal sequence modeling.
• Employed Grey Wolf Optimization (GWO) for efficient hyperparameter tuning and model configuration.
• Validated the model on benchmark datasets (ASL Alphabet, ASL MNIST) for static and dynamic sign classification.

Main Results:

• Achieved state-of-the-art performance with 99.40% accuracy, 99.31% F1-score, 0.988 MCC, and 0.992 AUC.
• Outperformed existing models including PCA-IGWO, KPCA-IGWO, GWO-CNN, and AEGWO-NET.
• Demonstrated robustness in real-time gesture detection across varied environmental conditions.

Conclusions:

• GWO-CTransNet provides a powerful and scalable solution for vision-based sign language recognition.
• The model offers high accuracy, fast inference, and adaptability for real-world assistive communication technologies.
• GWO integration significantly improved convergence speed and model generalization.