Ultrasound and Unsupervised Segmentation-Based Gesture Recognition for Smart Device Unlocking

Area of Science:

• Human-Computer Interaction
• Biometric Authentication
• Signal Processing

Background:

• Traditional device unlocking methods face security and privacy challenges.
• Gesture recognition offers a novel, intuitive authentication modality.
• Existing ultrasonic gesture recognition systems often lack real-time, on-device processing capabilities.

Purpose of the Study:

• To propose and validate a smart device unlocking scheme using ultrasonic gesture recognition.
• To develop a privacy-preserving, on-device authentication system.
• To achieve high accuracy and robustness in real-time gesture-based unlocking.

Main Methods:

• Utilized ultrasound to detect and analyze micro-features of multiple consecutive gestures.
• Employed unsupervised segmentation for accurate gesture feature extraction and time-frequency data analysis.
• Applied two-stage data enhancement and transfer learning for personalized, on-device model deployment.

Main Results:

• The authentication pipeline processed 8 gesture types in approximately 1.2 seconds, with model inference under 50 milliseconds.
• Achieved an average five-fold cross-validation accuracy of 93.56% for identity verification.
• Demonstrated robustness across different environmental conditions on a commercial smartphone.

Conclusions:

• The proposed ultrasonic gesture recognition system provides a secure, private, and efficient method for smart device unlocking.
• On-device, real-time inference ensures biometric data never leaves the user's device, enhancing privacy.
• The system's high accuracy and robustness make it a viable alternative to conventional authentication techniques.