Related Experiment Video
Updated: Sep 25, 2026

Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
Published on: March 28, 2025
Fine-grained multi-level gesture recognition based on a stretchable multichannel ultrasonic device
Xinyi Lin1,2, Hang Liu3, Kai Lin1
1Key Laboratory of Soft Machines and Smart Devices of Zhejiang Province, State Key Laboratory of Brain-Machine Intelligence, Department of Engineering Mechanics, Zhejiang University, Hangzhou 310027, China.
Abstract:
Discrete gesture recognition provides a direct output form for command-based human-machine interaction, while fine-grained multi-level recognition can expand command capacity by mapping subtle graded finger movements to distinct commands or different levels of the same command, thereby reducing the need for large or repetitive hand gestures. However, reliably distinguishing fine-grained multi-level gestures remains challenging. Here, we present a stretchable multichannel ultrasonic device comprising four functional sites and sixteen piezoelectric modules. Its fan-shaped substrate stretches up to 30%, enabling conformal forearm attachment and alignment with target muscle regions. Experimental characterization demonstrated sub-millimeter spatial resolution and excellent signal quality. Integrated with a one-dimensional convolutional neural network, the system achieved up to 98.75% accuracy in recognizing metacarpophalangeal joint-angle changes below 5°. The multichannel configuration yields high classification accuracy, improved class-wise recognition balance, more effective learning from multi-subject data and enhanced calibration-assisted adaptation to shifted device positions and new users, providing a reliable strategy for low-burden, fine-grained multi-level gesture command control.