Learning Frame-Level Classifiers for Video-Based Real-Time Assessment of Stroke Rehabilitation Exercises From Weakly...

Area of Science:

• Rehabilitation Engineering
• Computer Vision
• Machine Learning

Background:

• Virtual coaches require real-time feedback for effective stroke rehabilitation.
• Collecting fully annotated datasets for exercise assessment is costly and time-consuming.
• Current methods face challenges in evaluating real-time compensatory motion assessment.

Purpose of the Study:

• To develop a novel framework for real-time assessment of compensatory motions in stroke rehabilitation exercises.
• To enable frame-level classification using weakly annotated videos by generating pseudo-labels.
• To generalize to new exercises and patients with limited labeled data.

Main Methods:

• A baseline approach using a source dataset for frame-level classifier training.
• A transfer learning approach utilizing target video-level labels and source dataset parameters.
• A semi-supervised approach leveraging target video-level labels and a small set of frame-level labels.
• Pseudo-label generation for frame-level classification.

Main Results:

• The baseline approach showed limited generalization to a new dataset (SERE) with an F1-score of 72.87%.
• Transfer learning and semi-supervised approaches achieved higher F1-scores of 80.47% and 78.93%, respectively.
• Pseudo-label generation improved frame-level classification accuracy for the target dataset compared to the baseline.

Conclusions:

• The proposed framework effectively assesses compensatory motions in stroke rehabilitation exercises using weakly annotated videos.
• Transfer learning and semi-supervised methods significantly improve generalization to new datasets, exercises, and patients.
• This approach reduces data annotation effort, simplifying the customization of virtual rehabilitation coaches.