Interaction Detection in Egocentric Video: Toward a Novel Outcome Measure for Upper Extremity Function

Area of Science:

• Biomedical Engineering
• Rehabilitation Technology
• Computer Vision

Background:

• Effective rehabilitation for cervical spinal cord injury requires accurate hand function measurement.
• Current methods are limited to clinical settings, failing to capture real-world functional use.
• There is a need for unobtrusive, continuous monitoring of hand function in community environments.

Purpose of the Study:

• To demonstrate the feasibility of a wearable system for monitoring functional hand use.
• To develop and test computer vision algorithms for detecting hand-object interactions from egocentric video.
• To provide a tool for assessing hand function during daily activities post-injury.

Main Methods:

• A wearable system utilizing egocentric cameras to capture hand movements.
• Computer vision techniques including hand segmentation and feature extraction for interaction detection.
• A random forest classifier trained on activities of daily living and non-interactive tasks.

Main Results:

• The system achieved an average F-score of 0.85 on a custom dataset and 0.91 on a public dataset.
• Demonstrated feasibility in detecting hand-object interactions across various daily activities.
• Successful segmentation of the hand and extraction of relevant interaction features.

Conclusions:

• Egocentric video analysis is a viable method for monitoring functional hand use in community settings.
• This technology can support the development of effective upper extremity rehabilitation interventions.
• The proposed system offers a promising approach for remote patient monitoring in spinal cord injury recovery.