Ultrasound and Unsupervised Segmentation-Based Gesture Recognition for Smart Device Unlocking

A smart device unlocking scheme based on ultrasonic gesture recognition is proposed, allowing users to unlock their devices by customizing the unlock code through gesture movements. This method utilizes ultrasound to detect multiple consecutive gestures, identifying micro-features within these gestures for authentication. To enhance recognition accuracy, an unsupervised segmentation algorithm is employed to accurately segment the gesture feature region and extract the time-frequency domain data of the gestures. Additionally, two-stage data enhancement techniques are applied to generate user-specific data based on a small sample size. Finally, the user-specific model is deployed to mobile devices via transfer learning for on-device, real-time inference. Experimental validation on a commercial smartphone (Redmi K50) demonstrates that the entire authentication pipeline, from signal acquisition to decision, processes 8 types of gestures in a sequence in sequence in approximately 1.2 s, with the core model inference taking less than 50 milliseconds. This ensures that the raw biometric data (ultrasonic echoes) and the recognition results never leave the user's device during authentication, thereby safeguarding privacy. It is important to note that while model training is performed offline on a server to leverage greater computational resources for personalization, the deployed system operates fully in real time on the edge device. Experimental results demonstrate that our system achieves accurate and robust identity verification, with an average five-fold cross-validation accuracy rate of up to 93.56%, and it shows robustness across different environments.