Non-invasive assistive framework for sign language recognition using software-defined radio sensing and deep...

Area of Science:

• Assistive Technology
• Machine Learning
• Wireless Communication

Background:

• Traditional American Sign Language recognition (ASLR) methods using cameras or gloves have limitations in privacy and environmental adaptability.
• There is a need for non-invasive, privacy-preserving communication support for individuals with hearing impairments.
• Radio-frequency (RF) sensing offers a potential alternative for capturing human motion without direct visual or physical contact.

Purpose of the Study:

• To propose and evaluate a proof-of-concept framework for early-stage American Sign Language recognition (ASLR).
• To explore the technical feasibility of using RF sensing and time-series deep learning (DL) for ASLR.
• To develop a communication support mechanism that respects privacy and functions in diverse environmental conditions.

Main Methods:

• Utilized software-defined radio (SDR) with Universal Software Radio Peripheral (USRP) kits to transmit RF signals.
• Captured variations in wireless channel state information (WCSI) induced by human sign imprints.
• Processed and classified RF data using a DL architecture optimized for time-series analysis, trained on 20 American signs.

Main Results:

• Achieved 98% recognition accuracy for American signs using comparative analysis of time-series DL algorithms.
• Demonstrated the framework's robustness in nonintrusive operation and low-light conditions.
• Highlighted the interpretability of complex full-body movements captured through RF sensing.

Conclusions:

• The proposed framework shows potential for future communication-support technologies for individuals with hearing impairments.
• Integrating SDR frequency sensing with DL techniques can advance ASLR research.
• This approach lays the groundwork for scalable, privacy-preserving assistive communication systems.