A Graph-Based Multimodal Fusion Framework for Assessment of Freezing of Gait in Parkinson's Disease

Freezing of Gait (FOG) is a significant symptom contributing to gait dysfunction in Parkinson's disease (PD) patients. Most current methods for assessing FOG severity often overlook the interpretability of the extracted gait features. In this study, we design a multimodal gait feature dataset with rich physical significance, including kinematics, kinetics, and spatiotemporal modalities. We also propose a graph-based multimodal fusion framework (GMFF) to accurately quantify FOG severity. GMFF employs the graph attention mechanism to extract modality-specific features and utilizes the generalized canonical correlation analysis (GCCA) algorithm as the core of the feature fusion module. We provide the double-hurdle output module to address the impact of the zero-inflation problem on the performance of GMFF. We evaluate the performance of GMFF on a public PD gait database using five-fold cross-validation. The results demonstrate that GMFF achieves an accuracy of 0.978 in identifying patients with FOG and a root mean square error of 0.449 in quantifying FOG severity. Using the interpretability of GMFF, we identify the gait feature set that effectively characterizes the gait patterns of PD patients and then explore the impact of FOG symptoms on their walking ability under both the "ON" and "OFF" medication states. Thus, this study has the potential to provide valuable insights into the clinical monitoring and management of PD patients.