Privacy-Preserving British Sign Language Recognition Using Deep Learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Sign language, like British Sign Language (BSL), is a complex visual language crucial for deaf communication.
• Current camera-based sign language recognition systems face limitations including poor lighting, training challenges, and privacy concerns.
• There is a need for innovative, privacy-preserving methods for sign language recognition.

Purpose of the Study:

• To develop a contactless and privacy-preserving British Sign Language (BSL) recognition system.
• To explore the use of radar technology combined with deep learning for BSL emotion recognition.
• To identify six common emotions (confused, depressed, happy, hate, lonely, sad) within BSL gestures.

Main Methods:

• A novel system utilizing radar sensors to capture sign language data without physical contact.
• Data representation using spectrograms to visualize the captured radar signals.
• Application of state-of-the-art deep learning models (InceptionV3, VGG19, VGG16) for feature extraction and classification.

Main Results:

• The radar-based system successfully captured and processed BSL data.
• Deep learning models effectively extracted spatiotemporal features from spectrograms.
• The VGG16 model achieved a maximum classification accuracy of 93.33% across all considered emotion classes.

Conclusions:

• Radar technology offers a viable, contactless, and privacy-preserving alternative for sign language recognition.
• Deep learning models, particularly VGG16, demonstrate high efficacy in classifying BSL emotions from radar data.
• This proof-of-concept study paves the way for advanced BSL emotion recognition systems.