Non-invasive assistive framework for sign language recognition using software-defined radio sensing and deep learning

This study proposes an early-stage, non-invasive assistive framework as a proof-of-concept for American Sign Language recognition (ASLR) using radio-frequency (RF) sensing and time-series deep learning (DL) techniques. The framework aims to explore the technical feasibility of ASLR as a potential future support mechanism for communication between individuals with hearing impairments and the hearing community. Traditional approaches, such as using camera-based systems or wearable gloves, face limitations in privacy, environmental adaptability, and full-body motion capture.

We propose software-defined radio (SDR) sensing using Universal Software Radio Peripheral (USRP) kits to transmit RF signals and capture signal-induced variations in wireless channel state information (WCSI). These dynamic distortions generate unique time-series patterns in the RF data, which were processed and classified using a DL architecture optimised for time-series analysis. Data for 20 American signs were acquired via WCSI using an orthogonal frequency division multiplexing transceiver to measure human sign imprints.

Comparative analysis of time-series DL algorithms achieved 98% recognition accuracy, without compromising privacy. Key strengths of the proposed conceptual framework include robustness in nonintrusive operation and low-light conditions, as well as interpretability of complex full-body movements.

The findings suggest potential applicability for individuals with hearing impairments in future communication-support technologies, with possible implications for fostering more inclusive environments. Integrating SDR frequency sensing with advanced DL techniques, this work can provide preliminary evidence to advance ASLR research and lay the groundwork for future scalable, privacy-preserving assistive systems.