Data Knee Pads: A Lower Limb Motion Capture System Based on Heterogeneous Sensors

Accurate lower limb motion capture is crucial for improving performance and user experience in fields such as motion analysis, rehabilitation training, and virtual reality. Traditional motion capture systems can only provide motion information, and often face issues such as occlusion or drift due to the limitations of sensor characteristics. For this purpose, we have designed a new type of data knee pad that integrates an Inertial Measurement Unit (IMU) and five liquid metal sensors. IMU provides basic motion data, while liquid metal sensors can provide information on joint bending and muscle activity. In order to extract effective information from sensor signals, we have developed an pose estimation model. The model first uses Fast Fourier Transform (FFT) to perform time-domain and frequency-domain analysis on the signal, in order to reveal hidden features in the signal. Next, inverse FFT and feature extraction are performed using the Transformer encoder to capture key motion features in the signal. Finally, we utilize a fully connected regression network to transform the extracted features into reconstruction of lower limb movements. Our system's lower limb pose estimation performance has been validated through a series of experiments, with an average tracking error of 1.48° for the personalized model. In addition, the ability of the system to capture muscle activity signals was also verified through experiments. Our system has achieved high-precision measurement of knee joint bending angle while capturing muscle activity signals, which other existing technologies cannot achieve. This makes our system more widely applicable in fields such as motion detection and rehabilitation evaluation of muscle diseases.