Study on Auxiliary Rehabilitation System of Hand Function Based on Machine Learning with Visual Sensors

Area of Science:

• Rehabilitation Engineering
• Artificial Intelligence in Healthcare
• Neurorehabilitation

Background:

• Stroke survivors often face challenges in hand function recovery during mid-to-late Brunnstrom stages.
• Active patient participation is crucial for effective rehabilitation but can be difficult to maintain.
• Existing assessment methods may lack precision or engaging elements for home-based therapy.

Purpose of the Study:

• To assess hand function recovery in stroke patients using advanced technology.
• To develop an automated system for rehabilitation exercises encouraging active participation.
• To create a feasible and economical solution for home and community-based stroke rehabilitation.

Main Methods:

• Utilized a deep residual network (ResNet) with Focal Loss for gesture recognition (91.0% Macro F1 score).
• Employed Leap Motion 2 for precise hand tracking and established skeletal joint point mapping.
• Developed an augmented reality (AR) system via Unity with C# for real-time motion range quantification.

Main Results:

• Achieved high accuracy in gesture recognition (90.9% validation accuracy).
• Demonstrated technical feasibility and accuracy of the automated assessment and rehabilitation system.
• Collected a large static assessment gesture dataset (502,401 frames) based on the FMA scale.

Conclusions:

• The developed AI and AR system is technically feasible and accurate for stroke patient hand rehabilitation.
• The system has the potential to enhance motivation, interactivity, and self-efficacy in stroke survivors.
• This integrated framework provides a foundation for future clinical applications in neurorehabilitation.