Related Experiment Video
Updated: Jun 13, 2025
11:39
Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique
Published on: September 7, 2022
2.1K
Real-time sign language detection: Empowering the disabled community
Sumit Kumar1, Ruchi Rani2, Ulka Chaudhari2
1Symbiosis Institute of Technology, Pune Campus, Symbiosis International (Deemed University), Pune, Maharashtra 412115, India.
Methodsx
|September 9, 2024
Summary
This study introduces an accurate, real-time Indian Sign Language (ISL) recognition system using VGG16 with an attention mechanism. The innovative approach enhances communication for individuals with speaking and hearing disabilities without requiring sensors.
Area of Science:
- Computer Science
- Artificial Intelligence
- Human-Computer Interaction
Background:
- Communication barriers exist for individuals with speaking and hearing disabilities.
- Previous sign language recognition systems lacked accuracy and real-time capabilities.
- Indian Sign Language (ISL) classification presents unique challenges.
Purpose of the Study:
- To develop an accurate and real-time Indian Sign Language recognition system.
- To overcome limitations of prior deep learning approaches for sign language classification.
- To improve communication accessibility for the deaf and mute community.
Main Methods:
- Utilized a pre-trained VGG16 Convolutional Neural Network (CNN) with an attention mechanism.
- Trained the model using the Adam optimizer and cross-entropy loss function.
- Focused on classifying 23 distinct hand poses within Indian Sign Language.
Main Results:
- Achieved 97.5% accuracy with VGG16 and 99.8% accuracy with VGG16 plus attention mechanism for ISL classification.
- Demonstrated the effectiveness of transfer learning in ISL recognition.
- Developed a sensor-free, real-time sign language recognition system.
Conclusions:
- The VGG16 model with an attention mechanism significantly improves ISL recognition accuracy.
- The system offers a cost-effective and practical solution for real-time communication assistance.
- This technology empowers individuals with speaking and hearing disabilities by bridging communication gaps.
