Deep Learning Techniques for Spanish Sign Language Interpretation

Area of Science:

• Computer Science
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Hearing impairment affects approximately 5% of the global population, leading to communication barriers and social exclusion.
• Effective communication is crucial for individuals with hearing impairments to prevent social isolation and frustration.
• The Spanish sign language alphabet is vital for articulating proper nouns like names, streets, and trademarks.

Purpose of the Study:

• To develop and evaluate a system for interpreting the Spanish sign language alphabet.
• To facilitate communication for individuals with hearing impairments, particularly for proper nouns.
• To compare the effectiveness of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) for sign language interpretation.

Main Methods:

• An image dataset of the 30 signed letters of the Spanish alphabet was created.
• Two types of neural networks, CNNs and RNNs, were trained and compared for sign interpretation.
• The system was designed to handle both static and in-motion sign letters.

Main Results:

• A comparative analysis revealed the significance of spatial dimensions over temporal dimensions in sign interpretation.
• Convolutional Neural Networks (CNNs) demonstrated superior performance compared to Recurrent Neural Networks (RNNs).
• The maximum accuracy achieved by CNNs reached 96.42%.

Conclusions:

• The developed system effectively interprets the Spanish sign language alphabet, enhancing communication for the hearing impaired.
• CNNs are more suitable than RNNs for Spanish sign language alphabet recognition due to their ability to capture spatial features.
• This technology holds potential for reducing communication barriers and improving social inclusion for the deaf community.