Domain-aware Intermediate Pretraining for Dementia Detection with Limited Data.

Area of Science:

• Computational linguistics
• Artificial intelligence in healthcare
• Neuroscience

Background:

• Dementia detection via human speech shows promise but is hindered by limited data.
• General pretrained models (e.g., BERT) can improve dementia detection but are prone to overfitting when fine-tuned on small dementia datasets.

Purpose of the Study:

• To propose a domain-aware intermediate pretraining method to improve dementia detection from speech.
• To address the overfitting challenge in fine-tuning large models with limited dementia-specific data.

Main Methods:

• Utilized pseudo-perplexity to select domain-similar datasets for intermediate pretraining.
• Developed dataset-level and sample-level domain-aware intermediate pretraining techniques.
• Introduced IU-pseudo-perplexity, incorporating information units (IU) to reduce calculation complexity.

Main Results:

• Confirmed the effectiveness of perplexity in predicting model accuracy across 9 GLUE benchmark datasets.
• Demonstrated that domain-aware intermediate pretraining significantly improves dementia detection accuracy in most cases.
• Observed small differences in text-based perplexity between Alzheimer's Disease (AD) patients and Healthy Controls (HC).

Conclusions:

• Domain-aware intermediate pretraining is an effective strategy for enhancing dementia detection from speech.
• Incorporating acoustic features alongside text-based perplexity may further improve pretraining effectiveness.
• The proposed method offers a viable solution to the data scarcity problem in clinical AI applications.