WatchForce: Wrist-worn PPG and IMU sensing for hand force estimation

Area of Science:

• Human-Computer Interaction
• Wearable Technology
• Biomedical Engineering

Background:

• Realistic human-computer interaction requires accurate force estimation, a capability currently lacking in extended reality systems.
• Existing electromyography-based force estimation methods offer high accuracy but suffer from poor usability.
• Novel wearable systems are needed to overcome these limitations for advanced interactive technologies.

Purpose of the Study:

• To investigate the feasibility of using smartwatch-compatible photoplethysmography (PPG) and inertial measurement unit (IMU) signals for estimating hand forces.
• To evaluate the performance of a novel wrist-worn system, WatchForce, in classifying hand forces into three levels.
• To assess WatchForce's capability in regressing maximal voluntary contraction (MVC)-normalized force values.

Main Methods:

• Development of WatchForce, a wrist-worn system integrating PPG and IMU sensors.
• Implementation of a Vision Transformer-based model with a multimodal alignment strategy (bidirectional InfoNCE).
• Separate encoding and alignment of PPG and IMU signals for multimodal force estimation.

Main Results:

• WatchForce achieved 86.31% accuracy in force classification (session-level universal model) and 88.32% (user-dependent models).
• The regression task yielded an nRMSE of 0.1386 and an R² of 78.24% for MVC-normalized force.
• Leave-one-subject-out evaluation showed 78.50% classification accuracy and 0.1509 nRMSE/68.04% R² for regression, indicating generalization potential.

Conclusions:

• WatchForce successfully demonstrates the potential of smartwatch-compatible PPG and IMU signals for wrist-based hand-force estimation.
• The system shows promise for enhancing realistic interactions in extended reality and other human-computer interfaces.
• Future research should focus on larger, diverse cohorts and real-world validation using commercial smartwatch hardware.