Parkinson's Disease Prediction: An Attention-Based Multimodal Fusion Framework Using Handwriting and Clinical Data

Neurodegenerative diseases (NGD) encompass a range of progressive neurological conditions, such as Alzheimer's disease (AD) and Parkinson's disease (PD), characterised by the gradual deterioration of neuronal structure and function. This degeneration manifests as cognitive decline, movement impairment, and dementia. Our focus in this investigation is on PD, a neurodegenerative disorder characterized by the loss of dopamine-producing neurons in the brain, leading to motor disturbances. Early detection of PD is paramount for enhancing quality of life through timely intervention and tailored treatment. However, the subtle nature of initial symptoms, like slow movements, tremors, muscle rigidity, and psychological changes, often reduce daily task performance and complicate early diagnosis.

To assist medical professionals in timely diagnosis of PD, we introduce a cutting-edge Multimodal Diagnosis framework (PMMD). Based on deep learning techniques, the PMMD framework integrates imaging, handwriting, drawing, and clinical data to accurately detect PD. Notably, it incorporates cross-modal attention, a methodology previously unexplored within the area, which facilitates the modelling of interactions between different data modalities.

The proposed method exhibited an accuracy of 96% on the independent tests set. Comparative analysis against state-of-the-art models, along with an in-depth exploration of attention mechanisms, highlights the efficacy of PMMD in PD classification.

The obtained results highlight exciting new prospects for the use of handwriting as a biomarker, along with other information, for optimal model performance. PMMD's success in integrating diverse data sources through cross-modal attention underscores its potential as a robust diagnostic decision support tool for accurately diagnosing PD.