UCOPhyRehab++: A multi-modal and multi-view dataset for human rehabilitation analysis

The rehabilitation of patients with musculoskeletal disorders is usually associated with the performance of prescribed exercises at home. Performing these exercises without medical supervision may lead to incorrect execution, resulting in secondary injuries or slower recovery rates for these patients. For this reason, research into assisted rehabilitation methodologies for patients of this type has been one of the most studied fields in recent years. The use of computer vision techniques has rapidly increased in recent literature. However, there is a significant lack of available data for training machine learning models or for testing these systems. In this paper, we extend our previous work, UCOPhyRehab (University of COrdoba Physical Rehabilitation), by adding multiple modalities to the original data, incorporating demographic metadata, and including performance scores assigned by an expert physical therapist. Our validation experiments demonstrate that this new release complements the original UCOPhyRehab data and enables new research directions, such as multi-modal fusion (e.g., combining silhouettes, optical flow, and semantic segmentation) and multi-view fusion across the five camera viewpoints, to improve the robustness and accuracy of rehabilitation-assistance methods.