Robust Finger Interactions with COTS Smartwatches via Unsupervised Siamese Adaptation

Area of Science:

• Human-computer interaction
• Wearable technology
• Sensor fusion

Background:

• Smartwatches and wearable devices offer convenience but have limited interfaces.
• Small screens hinder user interaction and computing functionality.
• Existing solutions often require complex calibration or multiple sensors.

Purpose of the Study:

• To introduce ViWatch, a novel system for robust finger interaction on smartwatches.
• To overcome limitations of small smartwatch interfaces using ubiquitous IMU sensors.
• To develop an unsupervised learning method for accurate and adaptable interaction detection.

Main Methods:

• Designed an unsupervised Siamese adversarial learning framework.
• Utilized a single Inertial Measurement Unit (IMU) sensor common in smartwatches.
• Developed a real-time system deployed on commodity smartwatches.
• Conducted extensive testing with over 100 volunteers.

Main Results:

• Achieved approximately 97% system accuracy over a week-long evaluation.
• Demonstrated robustness against variations in hand shape, finger activity, and device placement.
• User study indicated a preference for the unsupervised approach over supervised calibration.

Conclusions:

• ViWatch provides a highly accurate and adaptable solution for finger-based interactions on smartwatches.
• The unsupervised learning method eliminates the need for tedious supervised calibration.
• ViWatch enhances smartwatch usability and opens possibilities for new mobile applications.