Speech-Derived Digital Biomarkers from a Brief Video-Recorded Dual-Task Test Differentiate Mild Cognitive Impairment:...

Dual-task testing, which combines a motor task with a simultaneous memory task, is commonly used to detect early cognitive decline because it challenges attention and executive function. However, most dual-task assessments emphasize motor performance while overlooking speech during the memory component. Speech reflects the integrity of multiple cognitive systems and has emerged as a promising digital biomarker of brain health. Yet, existing speech-based protocols typically rely on single-task, clinic-based recordings, limiting scalability and real-world applicability. Integrating speech analysis into brief, functional dual tasks may provide a practical solution for remote cognitive assessment.

To determine whether speech captured during a 20-second upper extremity frailty (UEF) dual-task test can distinguish individuals with mild cognitive impairment (MCI) from cognitively healthy adults in a fully remote environment.

In a multisite U.S. cohort, community-dwelling adults aged 50-79 years completed the Montreal Cognitive Assessment (MoCA) and a 20-second UEF dual task involving repeated elbow flexion-extension while counting backward aloud. Elbow motion and speech were recorded using the built-in webcam and microphone of a smartphone, tablet, or laptop during a Zoom session. Audio was extracted and transcribed into time-stamped text using an automated pipeline powered by OpenAI's Whisper large automatic speech recognition model. Custom-developed algorithms extracted nine speech-derived features reflecting timing, pausing behavior, and counting performance. Logistic regression models incorporating demographics, patient-reported outcomes, and speech features were constructed to classify cognitive status.

Among 319 recordings meeting predefined audio-quality criteria (110 MCI; 209 controls), multiple speech features differed significantly between groups (p<0.05) and showed modest correlations with MoCA scores (Spearman's ρ=0.26-0.32, p<0.05). Models including speech features significantly improved discrimination of MCI compared to demographics alone. The combined model achieved an area under the receiver operating characteristic curve of 0.79, compared with 0.56 for demographics-only models.

Speech captured during a brief, video-recorded dual-task test provides meaningful digital biomarkers of cognitive impairment. This device-agnostic protocol enables scalable, low-burden cognitive screening and supports longitudinal brain health monitoring outside traditional clinical settings.