Simultaneous Hand Gesture Classification and Finger Angle Estimation via a Novel Dual-Output Deep Learning Model

Area of Science:

• Human-Computer Interaction
• Machine Learning
• Biomedical Engineering

Background:

• Hand gesture classification and finger angle estimation are crucial for intuitive human-computer interaction (HCI).
• Existing methods often address these tasks in isolation, limiting their practical application.
• There is a need for integrated approaches that can simultaneously process both discrete gesture and continuous angle data.

Purpose of the Study:

• To develop and evaluate a dual-output deep learning model for simultaneous hand gesture classification and finger angle estimation.
• To investigate the effectiveness of using a wristband with barometric sensors for capturing spatial-temporal hand movement data.
• To compare the performance of the proposed integrated model against traditional, isolated learning approaches.

Main Methods:

• A dual-output deep learning model was designed to process spatial-temporal features extracted from sensor data.
• Data augmentation techniques were employed to enhance the training dataset.
• A wristband equipped with ten modified barometric sensors captured hand movement data from ten subjects performing various finger movements.
• Ground-truth finger angles were recorded using a data glove for model validation.

Main Results:

• The proposed model achieved a high hand gesture classification accuracy of 97.5%.
• Finger angle estimation yielded a coefficient of determination (R²) of 0.922.
• These results demonstrated significantly superior performance compared to shallow learning approaches applied in isolation.

Conclusions:

• The developed dual-output deep learning model effectively enables simultaneous hand gesture classification and finger angle estimation.
• This integrated approach offers a promising solution for enhancing intuitive human-computer interaction.
• The method is suitable for applications requiring the interpretation of both discrete and continuous human movement variables.