Validation Study on Movement Sonification of the Hand as a Rehabilitation Tool

Introduction: Hand injuries can lead to lasting impairments that limit one's ability to perform activities of daily living. Many individuals who undergo surgery following a hand injury participate in post-operative rehabilitation. Evidence suggests that geographical and socioeconomic barriers may hinder rehabilitation progress and limit recovery after surgery. Telemedicine and augmented reality may be useful tools for addressing issues concerning accessibility and adherence in rehabilitation. This study validates the use of a novel movement sonification interface that uses Google's MediaPipe hand-tracking technology to map single-digit PIP joint motion to real-time auditory feedback. Methods: Nineteen healthy adults (mean age 32 years; 10 males, 9 females) with no prior exposure to movement sonification participated. Through webcam, participants interacted with a web-based program that mapped fourth digit PIP joint flexion and extension to musical pitch changes (low pitch for extension, high pitch for flexion). Construct, face, content, and predictive criterion validity were assessed through structured surveys after participants viewed and explored the program. Results: Construct validity was 89%, with participants correctly identifying the relationship between fourth digit PIP movement and pitch changes. Both face and content validity were 100%, with participants recognizing the sound-movement relationship and that only the fourth digit produced sound. Predictive criterion validity demonstrated high accuracy, with participants correctly matching hand configurations to sounds (95%) and predicting sounds from muted videos (95%). Conclusions: Results demonstrate excellent construct, face, content, and predictive criterion validity. This supports the feasibility of using music sonification as an intuitive and accessible tool for augmented hand rehabilitation.