Related Experiment Video
Updated: Jun 26, 2025
08:15
Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
Published on: March 28, 2025
455
End-to-End Ultrasonic Hand Gesture Recognition.
Elfi Fertl1,2, Do Dinh Tan Nguyen1, Martin Krueger1
1Infineon Technologies AG, 85579 Neubiberg, Germany.
Sensors (Basel, Switzerland)
|May 11, 2024
Summary
This study introduces a new machine learning approach for ultrasound-based hand gesture recognition (HGR) using low-cost sensors. The method achieves high accuracy by directly processing raw echo data, simplifying the process for efficient human-computer interaction.
Area of Science:
- Human-Computer Interaction
- Signal Processing
- Machine Learning
Background:
- Increasing demand for intuitive input methods in electronic devices.
- Limitations in robustness and accuracy of current ultrasound-based hand gesture recognition (HGR) systems.
- Need for cost-effective and power-efficient HGR solutions.
Purpose of the Study:
- To present a novel machine learning (ML)-based end-to-end solution for HGR.
- To utilize low-cost micro-electromechanical (MEMS) system ultrasonic transducers.
- To demonstrate high accuracy with minimal data pre-processing.
Main Methods:
- Development of a custom hardware setup with four MEMS ultrasonic transducers in various arrangements.
- Direct processing of raw ultrasonic echo samples by ML models.
- Benchmarking of various ML models including CNNs, GRUs, LSTMs, ViTs, and CrossViTs.
Main Results:
- Achieved over 88% accuracy with LSTM, ViT, and CrossViT models.
- Demonstrated that minimal pre-processing, even omitting Fourier transforms, yields high accuracy.
- Showcased the effectiveness of direct raw echo sample processing by ML models.
Conclusions:
- Minimal pre-processing is sufficient for high-accuracy ultrasonic HGR using cost-effective MEMS transducers.
- Direct signal processing by compact ML models enables low-cost, power-efficient HGR.
- This approach offers a viable alternative to existing HGR systems using vision, WiFi, or radar.
