Frame-Level Real-Time Assessment of Stroke Rehabilitation Exercises from Video-Level Labeled Data: Task-Specific vs....

Area of Science:

• Biomedical Engineering
• Rehabilitation Technology
• Artificial Intelligence in Healthcare

Background:

• Stroke rehabilitation requires continuous patient exercise and feedback.
• Virtual coaches offer potential for autonomous exercising and motor function improvement.
• Current motion analysis systems need extensive frame-level annotations, hindering scalability.

Purpose of the Study:

• To develop a framework for real-time assessment of compensatory motions in stroke rehabilitation exercises using video-level annotations.
• To reduce the reliance on costly and time-consuming frame-level data labeling.
• To enhance the generalization ability of motion analysis models for new patients.

Main Methods:

• A gradient-based technique and pseudo-label selection were used to generate frame-level labels from video-level annotations.
• Pre-trained models including Action Transformer, SkateFormer, and MOMENT were leveraged for pseudo-label generation.
• The SERE dataset, comprising 18 post-stroke patients, was utilized for validation.

Main Results:

• The MOMENT foundation model achieved superior video-level assessment with an AUC of 73%, significantly outperforming the baseline LSTM (AUC = 58%).
• The Action Transformer model, combined with the Integrated Gradient technique, yielded improved frame-level assessment (AUC = 72%) compared to frame-level ground truth labeling (AUC = 69%).
• The proposed approach demonstrated enhanced model generalization and customization for new patients.

Conclusions:

• The developed framework effectively enables frame-level motion classification from video-level annotations for stroke rehabilitation.
• Leveraging pre-trained models and pseudo-labeling significantly reduces data annotation burden and improves model performance.
• This approach facilitates the development of more accessible and adaptable virtual coaching systems for stroke recovery.