Multi-Modal Feature Fusion Using Transformer for Early Alzheimer's Disease Diagnosis

Alzheimer's disease (AD) is a common neurodegenerative disorder. Early and accurate diagnosis of AD is essential for effective treatment. However, due to the class imbalance problem, there is a significant data gap between different categories. Moreover,the data feature differences of AD are relatively small, which poses challenges for its application in the early diagnosis of AD. To tackle these problems, we propose an intelligent early AD diagnosis model based on Transformer. The deep learning diagnosis model utilizes Transformer to integrate image features and non-image features. Furthermore, it incorporates a class imbalance loss function to optimize the performance of early AD diagnosis, thereby enhancing the model's ability to recognize underrepresented classes. In order to alleviate the problem of class imbalance and test the model performance, we used stratified 5-fold cross validation to verify the model effect.Experimental results demonstrate that our model can significantly improve the accuracy of AD diagnosis, which is markedly better than traditional methods. Additionally, loss function we used more effectively mitigates the problem of class imbalance. We believe this work can effectively reduce the burden on medical staff to diagnose early AD.