WavBERT: Exploiting Semantic and Non-semantic Speech using Wav2vec and BERT for Dementia Detection

Area of Science:

• Artificial Intelligence
• Computational Linguistics
• Medical Informatics

Background:

• Dementia detection often relies on cognitive tests and clinical assessments.
• Analyzing speech patterns offers a non-invasive method for early dementia detection.
• Existing methods may not fully capture the nuances within speech data.

Purpose of the Study:

• To develop and evaluate advanced speech analysis models for dementia detection.
• To integrate both semantic and non-semantic speech information for improved diagnostic accuracy.
• To introduce the WavBERT model, leveraging Wav2vec and BERT for enhanced dementia detection.

Main Methods:

• Utilized Wav2vec for extracting semantic features from patient speech.
• Employed Bidirectional Encoder Representations from Transformers (BERT) for analyzing semantic information.
• Developed extended WavBERT models to incorporate non-semantic speech features, including pause analysis.
• Designed a pre-trained embedding conversion network for fine-tuning WavBERT with non-semantic data.

Main Results:

• WavBERT models achieved 83.1% accuracy in dementia classification.
• The models obtained a low Root-Mean-Square Error (RMSE) of 4.44 in regression tasks.
• A mean F1 score of 70.91% was recorded for the progression task.
• Demonstrated the effectiveness of incorporating both semantic and non-semantic speech information.

Conclusions:

• The WavBERT models show significant promise for accurate and effective dementia detection.
• Integrating non-semantic speech features enhances the performance of AI-based diagnostic tools.
• Speech analysis using advanced AI models offers a valuable avenue for early dementia identification.