Fuzzy Synchronization Likelihood Graph in Deep Neural Networks for Human Motion Time Series Analysis

Variable interactivity is crucial in biological multivariate time series analysis. This research suggests using graph structures to represent such interactions for more explainable decision-making processes. However, measuring the variable interaction in a graph is an open problem with no unique solution. Existing graph construction methods are either computationally costly, require extensive training, or disregard the inherent data nonlinearities and nonstationarity. We propose using the Fuzzy Synchronization Likelihood (FSL) criterion to address these challenges in constructing a graph and examining the qualitative similarity and dependency among variables. We propose applying this strategy to automated rehabilitation exercise evaluations based on human joint motion data. This multivariate time series application benefits from FSL-constructed graphs by offering further insight into the kinematics of joint interactions. Finally, we extend the convolutional layer in the Deep Mixture Density Neural Network (DMDN) to process the FSL-constructed graph, extracting practical information regarding task-based variable dependencies. An ablation study shows that the proposed FSL Graph-based Deep Neural Network (FSLGDN) outperforms its competitive approaches that use linear correlations and human anatomy for graph construction. Results also indicate that task-based consideration of joint motion data interactions is more beneficial than anatomy. Furthermore, the inherent nonstationarity of motion data leads to the extraction of more information than its linear correlation counterpart. Finally, while the proposed approach ranks competitively with a DMDN, the proposed approach's graph construction and representation of feature dependencies are more intuitive, leading to more explainable decision-making processes.