Exploring Deep Transfer Learning Techniques for Alzheimer's Dementia Detection | JoVE Visualize

Area of Science:

Computational linguistics
Artificial intelligence
Neuroscience

Background:

Speech analysis reveals links between vocal patterns and cognitive abilities, including dementia.
Collecting clinical speech and baseline data for dementia research is costly, leading to limited datasets.
Transfer learning offers a solution by utilizing knowledge from larger, related datasets to mitigate data scarcity.

Purpose of the Study:

To investigate the efficacy of deep transfer learning techniques for dementia detection using spontaneous speech.
To compare the performance of various pre-trained models (MobileNet, YAMNet, Mockingjay, BERT) across different data modalities (image, audio, speech, text).
To analyze the impact of multi-modal and multi-task transfer learning on dementia detection accuracy and regression.

Main Methods:

Utilized the ADReSS challenge spontaneous speech dataset, specifically the Cookie Theft Picture (CTP) task, with balanced participant groups.
Implemented deep transfer learning models including MobileNet (image), YAMNet (audio), Mockingjay (speech), and BERT (text).
Evaluated single-modal, multi-modal, and multi-task transfer learning approaches for dementia classification and regression.

Main Results:

Text-based transfer learning models (BERT) demonstrated significantly superior performance compared to audio-based models (YAMNet).
Multi-modal transfer learning showed marginal improvements, indicating limited complementary information between audio and text data.
Multi-task transfer learning yielded minimal gains in classification and a negative impact on regression, potentially due to label/score inconsistencies.

Conclusions:

Deep transfer learning, particularly with text data, is a promising approach for dementia detection from speech, effectively addressing data limitations.
Pre-trained text models show strong potential due to the inherent similarities between their training data and speech-derived text.
Further research into multi-task learning is needed, considering potential inconsistencies between diagnostic labels and cognitive scores.