Speaker-turn aware diarization for speech-based cognitive assessments.

Area of Science:

• Speech processing
• Cognitive science
• Biomedical engineering

Background:

• Speaker diarization is crucial for analyzing speech in cognitive assessments like the Montreal cognitive assessment (MoCA).
• Conventional methods face challenges with acoustic mismatch and accurately identifying speaker turns in MoCA recordings.

Purpose of the Study:

• To propose and evaluate three enhancements for speaker diarization in MoCA recordings.
• To improve diarization accuracy and robustness against various mismatch conditions.
• To enable speaker-turn timestamp hypothesization for datasets lacking this information.

Main Methods:

• Utilizing multi-scale channel interdependence speaker embedding with Res2Net blocks, squeeze-and-excitation units, and channel-dependent attention.
• Applying a sequence comparison approach for a holistic conversational view to create a speaker-turn aware scoring matrix.
• Incorporating a pairwise similarity measure to integrate local and global information into the scoring matrix.

Main Results:

• The proposed enhancements significantly outperform conventional x-vector/PLDA systems in speaker diarization.
• Demonstrated superior performance under language-, age-, and microphone-mismatch scenarios.
• Successfully hypothesized speaker-turn timestamps, enhancing applicability to diverse datasets.

Conclusions:

• The developed speaker diarization system offers improved accuracy and robustness for cognitive impairment diagnosis from speech.
• The method's ability to hypothesize speaker turns makes it valuable for analyzing speech datasets, even those without explicit timestamp information.