Synergy-Based Evaluation of Hand Motor Function in Object Handling Using Virtual and Mixed Realities

Area of Science:

• Rehabilitation Engineering
• Human-Computer Interaction
• Motor Control

Background:

• Traditional hand function assessments are subjective and lack detailed motor coordination insights.
• Limitations exist in current objective measures for fine motor skill evaluation.
• Need for advanced tools to analyze complex hand movements in clinical settings.

Purpose of the Study:

• To introduce a novel system for objective hand motor function evaluation using synergy-based analysis.
• To develop a system integrating virtual and mixed-reality environments for naturalistic object manipulation tasks.
• To demonstrate the capability of synergy-based analysis in detecting subtle motor abnormalities.

Main Methods:

• Utilized the Leap Motion Controller (LMC) for non-intrusive finger motion capture.
• Applied Principal Component Analysis (PCA) with Varimax rotation for spatial synergy extraction.
• Incorporated HoloLens 2 for a mixed-reality object manipulation task to enhance user spatial awareness.

Main Results:

• Synergy-based analysis revealed hand movement abnormalities not detectable by conventional task performance metrics.
• The system successfully captured detailed finger coordination patterns during object manipulation.
• Mixed-reality integration improved user immersion and natural interaction with virtual objects.

Conclusions:

• The developed system offers a more objective and detailed evaluation of finger motor function compared to traditional methods.
• Synergy-based analysis shows promise for personalized rehabilitation strategies and early motor impairment detection.
• This approach advances the field of motor function assessment in virtual and mixed-reality environments.