An interactive game for rehabilitation based on real-time hand gesture recognition

Area of Science:

• Biomedical Engineering
• Artificial Intelligence
• Rehabilitation Medicine

Background:

• Cardiovascular and cerebrovascular diseases pose significant global health risks, often leading to post-surgical motor or cognitive impairments.
• Human-computer interactive systems, particularly those leveraging artificial intelligence (AI), are emerging as vital tools for innovative rehabilitation therapies.
• Effective rehabilitation is crucial for improving patient outcomes and quality of life after these debilitating conditions.

Purpose of the Study:

• To develop an interactive game system to aid rehabilitation exercises for patients with cardiovascular and cerebrovascular diseases.
• To enhance hand-eye coordination through a game-like experience, making rehabilitation more engaging.
• To introduce a novel, lightweight residual graph convolutional architecture for real-time hand gesture recognition.

Main Methods:

• A lightweight residual graph convolutional architecture was proposed for accurate hand gesture recognition.
• An interactive system was designed integrating the proposed gesture recognition module with third-party components.
• Real-time skeleton-based hand gesture recognition was utilized for user interaction within the game.

Main Results:

• The developed system successfully recognized hand gestures in real-time for interactive gameplay.
• Participant testing demonstrated that most users improved their game passing rate during the evaluation period.
• The system showed promise in enhancing hand-eye coordination, a key aspect of rehabilitation.

Conclusions:

• The proposed AI-driven interactive game system shows potential as an effective tool for rehabilitation, particularly for improving hand-eye coordination.
• The lightweight residual graph convolutional architecture is suitable for real-time hand gesture recognition in interactive rehabilitation applications.
• Further development and clinical validation could establish this system as a valuable adjunct to traditional therapies for patients recovering from cardiovascular and cerebrovascular events.