Related Experiment Video
Updated: Sep 17, 2025
Electroencephalographic Signal Acquisition Framework for Neurodiverse: A Case Study of Dolphin-Assisted Therapy
Published on: June 27, 2025
Gesture recognition for hearing impaired people using an ensemble of deep learning models with improving beluga whale
Mohammed Assiri1,2, Mahmoud M Selim3,4
1Department of Computer Science, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University, P.O. BOX 16273, 3963, Al-Kharj, Saudi Arabia.
This study introduces a new model for recognizing sign language gestures, achieving 98.72% accuracy. The Gesture Recognition for Hearing Impaired People Using an Ensemble of Deep Learning Models with Improving Beluga Whale Optimization (GRHIP-EDLIBWO) model enhances communication for individuals with hearing impairments.
Area of Science:
- Computer Science
- Artificial Intelligence
- Human-Computer Interaction
Background:
- Sign language (SL) is crucial for communication among hearing-impaired individuals.
- Current machine learning models face challenges in directly understanding human language, necessitating effective gesture recognition systems.
- Gesture recognition (GR) systems bridge the gap, enabling human-machine interaction and benefiting the hearing-impaired and elderly.
Purpose of the Study:
- To propose an advanced model, GRHIP-EDLIBWO, for accurate sign language gesture recognition.
- To develop an accessible communication system for hearing-impaired individuals.
- To improve the performance of gesture recognition using deep learning and optimization techniques.
Main Methods:
- Image preprocessing using Sobel filter (SF) for edge detection and feature extraction.
- Feature extraction utilizing squeeze-and-excitation capsule network (SE-CapsNet) for spatial hierarchies.
- Ensemble classification with bidirectional gated recurrent unit (BiGRU), Variational Autoencoder (VAE), and bidirectional long short-term memory (BiLSTM), optimized by improved beluga whale optimization (IBWO).
Main Results:
- The GRHIP-EDLIBWO model achieved a superior classification accuracy of 98.72% on an Indian Sign Language (ISL) dataset.
- Extensive simulations demonstrated the model's robustness and effectiveness.
- The proposed method outperformed existing gesture recognition models.
Conclusions:
- The GRHIP-EDLIBWO model offers a significant advancement in sign language recognition technology.
- This approach can facilitate the development of more accessible communication tools for the hearing-impaired community.
- The integration of deep learning and optimization techniques shows great promise for future human-computer interaction research.
