Personalized Video-Based Hand Taxonomy Using Egocentric Video in the Wild.

Area of Science:

• Human-Computer Interaction
• Robotics
• Rehabilitation Engineering

Background:

• Hand function is critical for environmental interaction.
• Existing hand grasp models lack personalization, especially for individuals with impaired hands like those with spinal cord injuries (SCI).
• Varied sensorimotor impairments in SCI necessitate individualized analysis methods for grasping strategies.

Purpose of the Study:

• To automatically identify dominant, distinct hand grasps for individuals without predefined taxonomies.
• To create personalized hand taxonomies using egocentric video data.
• To analyze unique grasping techniques developed by individuals with SCI.

Main Methods:

• Applied semantic clustering to egocentric video recordings from 19 individuals with cervical SCI.
• Utilized a deep learning model integrating posture and appearance data.
• Focused on clustering grasping actions with semantic significance in naturalistic settings.

Main Results:

• Achieved a cluster purity of 67.6% ± 24.2% with 18.0% ± 21.8% redundancy.
• Qualitative assessment confirmed meaningful clusters within the video data.
• Demonstrated the ability to generate personalized hand taxonomies.

Conclusions:

• The proposed methodology offers a flexible and effective strategy for analyzing hand function in real-world environments.
• This approach has significant applications in clinical assessment and characterizing human-environment interactions.
• Enables in-depth understanding of diverse grasping behaviors, particularly in populations with motor impairments.