Using deep learning to detect upper limb compensation in individuals post-stroke using consumer-grade webcams-A...

Area of Science:

• Biomedical Engineering
• Rehabilitation Science
• Computer Vision

Background:

• Stroke rehabilitation requires effective upper limb assessments, ideally home-based.
• Computer vision and pose estimation offer potential for remote movement analysis.
• Compensatory movements are common in stroke survivors and impact function.

Purpose of the Study:

• To investigate the use of webcam-based human pose estimation and deep learning for automatic detection of compensatory movements during a drinking task in stroke survivors.
• To evaluate factors influencing detection accuracy, including data representation and model architecture.
• To assess the potential for personalized, at-home rehabilitation tools.

Main Methods:

• Twenty participants with stroke performed a drinking task, recorded via multiple cameras and optical motion capture (OMC).
• Therapists labeled compensatory movements; human poses were extracted using MediaPipe.
• Deep learning models were trained to predict compensatory movements using raw keypoints and custom features.

Main Results:

• Inter-person compensation detection accuracy reached 70% with custom features, but generalization was limited.
• Intra-person classification accuracy exceeded 90%.
• OMC data significantly improved accuracy; CNNs outperformed LSTMs. Limitations include pose estimation uncertainty and data variability.

Conclusions:

• Deep learning can differentiate compensatory from non-compensatory movements with accurate representations.
• Personalized models using consumer cameras show potential for supporting home-based stroke rehabilitation.
• Further improvements in pose estimation and dataset diversity are needed for robust generalization.