Multimodal deep learning for dementia classification using text and audio.

Area of Science:

• Neurology
• Computer Science
• Artificial Intelligence

Background:

• Dementia is a progressive neurological disorder impacting older adults' communication and cognition.
• Early diagnosis of dementia is crucial for improving quality of life and lifespan.
• Automated machine learning offers potential for enhanced dementia screening.

Purpose of the Study:

• To develop and evaluate deep learning models for automated dementia classification.
• To investigate the impact of text-based data augmentation on model performance.
• To assess the utility of audio and timestamp data in dementia classification.

Main Methods:

• Utilized the Pitt Cookie Theft dataset from DementiaBank for binary classification of dementia.
• Fine-tuned Wav2vec (audio) and Word2vec (text) models.
• Experimented with original data, data excluding short sentences, and text-augmented versions.

Main Results:

• Synonym-based text data augmentation substantially improved text-based model performance.
• Augmented text models achieved ~80% accuracy and ~90% AUROC, compared to ~60% accuracy and ~70% AUROC for non-augmented models.
• Audio or timestamp data did not yield significant performance improvements.

Conclusions:

• Text-based data augmentation is effective for improving automated dementia classification models.
• Deep learning models, particularly with text augmentation, show potential for dementia screening.
• Further research can explore multimodal approaches and refined data augmentation techniques.