Integrating Fine-Tuned LLM With Acoustic Features for Enhanced Detection of Alzheimer's Disease

Dementia represents a global public health concern, with the early detection of Alzheimer's disease, the most prevalent form of dementia, being of paramount importance. Given the limited availability of suitable biomarkers, research has shown that early cognitive impairment can be identified through patients' spoken language. This paper presents a multi-modal system for automatic Alzheimer's disease detection using speech. The system has been trained on spoken recordings of healthy individuals and Alzheimer's patients describing an image, a task requiring linguistic and cognitive skills. Built on fine-tuned advanced Large Language Models, audio feature extractors, and classifiers, the system, after an extensive comparison of single and multi-modal architectures, achieves optimal results with the combination of Mistral-7B, VGGish, and Support Vector Classifier, outperforming previous methods on the ADReSSo 2021 test set.