Sign language recognition using modified deep learning network and hybrid optimization: a hybrid optimizer (HO) based...

Area of Science:

• Artificial Intelligence
• Computer Vision
• Human-Computer Interaction

Background:

• Speech impairment hinders oral and auditory communication.
• Real-time sign language recognition (SLR) is crucial for bridging communication gaps.
• Developing accurate and continuous SLR models remains challenging due to variations in signers' speed and duration.

Purpose of the Study:

• To develop an advanced deep learning model for accurate and continuous sign language recognition.
• To address the challenges of signer independence and variability in sign language.
• To improve the performance of sign language detection systems.

Main Methods:

• Utilized Visual Geometry Group 16 (VGG16) for spatial and geometric feature extraction.
• Employed optical flow for motion feature extraction.
• Developed a novel deep learning model, CNNSa-LSTM, combining Convolutional Neural Network (CNN), Self-Attention (SA), and Long-Short-Term Memory (LSTM).
• Introduced a Hybrid Optimizer (HO) integrating the Hippopotamus Optimization Algorithm (HOA) and Pathfinder Algorithm (PFA).

Main Results:

• The proposed CNNSa-LSTM model achieved a high accuracy of 98.7%.
• Demonstrated superior performance with sensitivity at 98.2% and precision at 98.5%.
• Achieved a low Word Error Rate (WER) of 0.131 and Sign Error Rate (SER) of 0.114.
• Obtained a Normalized Discounted Cumulative Gain (NDCG) of 98%.

Conclusions:

• The CNNSa-LSTM model effectively processes complex, sequential data for sign language recognition.
• The hybrid optimization approach enhances model performance and accuracy.
• This research offers a significant advancement in real-time sign language detection technology.