Related Experiment Video
Updated: Nov 7, 2025
09:41
Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping
Published on: April 21, 2023
1.9K
Deep Residual Networks for User Authentication via Hand-Object Manipulations
Kanghae Choi1, Hokyoung Ryu1, Jieun Kim1
1ImagineX Lab, Graduate School of Technology and Innovation Management, Hanyang University, Seoul 04763, Korea.
Sensors (Basel, Switzerland)
|April 30, 2021
Summary
This study introduces a new method for continuous user authentication using hand-object manipulation behaviors captured by inertial measurement units (IMUs). Deep learning models achieved high accuracy in identifying users, enhancing security for wearable devices.
Area of Science:
- Biometrics and Human-Computer Interaction
- Machine Learning and Deep Learning
- Wearable Technology and Security
Background:
- Wearable devices enable continuous user authentication through behavioral biometrics.
- Authentication methods leveraging complex hand behaviors remain underexplored.
- Existing methods often lack ecological validity in real-world scenarios.
Purpose of the Study:
- To develop an implicit and continuous user authentication model based on hand-object manipulation behavior.
- To evaluate the effectiveness of deep learning models, specifically Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), for this authentication task.
- To assess the model's performance across different age groups and object manipulation complexity.
Main Methods:
- Utilized a finger- and hand-mounted inertial measurement unit (IMU)-based system to collect hand-object manipulation data.
- Employed state-of-the-art deep learning models: three Convolutional Neural Network (CNN)-based Deep Residual Networks (ResNets) (50, 101, 152 layers) and two Recurrent Neural Network (RNN)-based Long Short-Term Memory (LSTM) networks (simple and bidirectional).
- Collected data across three age groups and two scenarios (simple and complex daily object manipulation) to enhance ecological validity.
Main Results:
- Both ResNets and LSTMs models demonstrated acceptable performance in identifying users' hand behavior patterns, achieving an average accuracy of 96.31% and an F1-score of 88.08%.
- In simple hand behavior scenarios, deeper ResNets (ResNet-152 > ResNet-101 > ResNet-50) showed improved performance.
- ResNet models outperformed LSTMs in complex hand behavior scenarios, with the ResNet-152 achieving a false rejection rate of 8.34% and an equal error rate of 1.62%.
Conclusions:
- Deep learning models, particularly ResNets, are effective for implicit and continuous user authentication based on hand-object manipulation.
- The proposed system demonstrates high accuracy and potential for real-world application in securing wearable devices.
- Deeper network architectures can enhance performance in complex behavioral authentication tasks without significant degradation.
