End-to-End Ultrasonic Hand Gesture Recognition.

Area of Science:

• Human-Computer Interaction
• Signal Processing
• Machine Learning

Background:

• Increasing demand for intuitive input methods in electronic devices.
• Limitations in robustness and accuracy of current ultrasound-based hand gesture recognition (HGR) systems.
• Need for cost-effective and power-efficient HGR solutions.

Purpose of the Study:

• To present a novel machine learning (ML)-based end-to-end solution for HGR.
• To utilize low-cost micro-electromechanical (MEMS) system ultrasonic transducers.
• To demonstrate high accuracy with minimal data pre-processing.

Main Methods:

• Development of a custom hardware setup with four MEMS ultrasonic transducers in various arrangements.
• Direct processing of raw ultrasonic echo samples by ML models.
• Benchmarking of various ML models including CNNs, GRUs, LSTMs, ViTs, and CrossViTs.

Main Results:

• Achieved over 88% accuracy with LSTM, ViT, and CrossViT models.
• Demonstrated that minimal pre-processing, even omitting Fourier transforms, yields high accuracy.
• Showcased the effectiveness of direct raw echo sample processing by ML models.

Conclusions:

• Minimal pre-processing is sufficient for high-accuracy ultrasonic HGR using cost-effective MEMS transducers.
• Direct signal processing by compact ML models enables low-cost, power-efficient HGR.
• This approach offers a viable alternative to existing HGR systems using vision, WiFi, or radar.