Convolutional Neural Network Based Real Time Arabic Speech Recognition to Arabic Braille for Hearing and Visually...

Area of Science:

• Artificial Intelligence
• Assistive Technology
• Human-Computer Interaction

Background:

• Hearing and visually impaired individuals face significant information deficits and communication barriers.
• Existing assistive technologies may not adequately address the complex needs of those with combined hearing and visual impairments.
• Bridging the information gap is crucial for the independence and well-being of individuals with dual sensory loss.

Purpose of the Study:

• To develop a real-time Arabic speech recognition system for the hearing and visually impaired.
• To translate recognized Arabic speech into Arabic Braille for tactile feedback.
• To enhance communication and information accessibility for individuals with combined hearing and visual impairments.

Main Methods:

• Utilized Convolutional Neural Network (CNN) algorithms for real-time Arabic speech recognition.
• Converted recognized Arabic text into Arabic Braille characters.
• Controlled a Braille display with a solenoid drive for tactile output, deciphered by users proficient in Braille.

Main Results:

• Achieved 90% model training accuracy using CNN with ReLU activation function.
• Demonstrated an 84% improvement in recognition accuracy with ReLU activation in testing.
• Successfully enabled tactile communication of Arabic digits via a Braille display for participants.

Conclusions:

• The developed CNN-based system effectively converts Arabic speech to Braille, offering a novel communication channel.
• This technology significantly improves information access for the hearing and visually impaired.
• The system holds promise for fostering greater independence and social integration for individuals with dual sensory impairments.