Predicting subjective sleepiness during auditory cognitive testing using voice signaling analysis

To determine whether objective markers of sleepiness can be collected passively using voice data to detect sleepiness in individuals undergoing testing in situations where sleepiness is not the focal point of assessment. We assessed verbal reaction time (VRT) as a vocalic marker of subjective sleepiness in middle aged and older adults with history of insomnia and benzodiazepine-receptor-agonist (BZRA) use.

Adults aged ≥55 without a diagnosis of dementia were recruited from a BZRA deprescribing clinical trial and enrolled in the present study that tested the feasibility of cognitive testing using out-of-office, self-directed mobile apps. Participants' working/episodic memory were assessed through recorded verbal responses to Verbal Paired Associates (VPA) tests, and ecological momentary assessments (EMA) of self-reported sleepiness (1[not at all] to 4[more prominent]). Using a generalized additive model, we examined the association between VRT during VPA testing and self-reported sleepiness, adjusting for demographic, test parameters, caffeine intake, cognition, mood, and BZRA-use (p≤0.05 was considered significant). A stratified k-fold cross-validation/random forest (SKCV/RF) was performed to classify sleepiness levels, adjusting for other variables.

We analyzed 1,513 observations from 16 patients. VRT was operationalized as the time duration between recording start time and first speech epoch. Longer VRTs were positively associated with greater EMA sleepiness (p≤0.05). The SKCV/RF model yielded a mean F1-score of 0.80 ± 0.08 across folds.

Longer VRTs correlated with greater self-reported sleepiness, indicating that voice data can be used as a marker of sleepiness in patients undergoing cognitive testing in out-of-office settings.

The online version contains supplementary material available at 10.1186/s41606-025-00141-y.