SPEECH TASKS RELEVANT TO SLEEPINESS DETERMINED WITH DEEP TRANSFER LEARNING.

Area of Science:

• Speech analysis
• Machine learning
• Sleep science

Background:

• Excessive sleepiness poses risks in critical situations like driving.
• Current sleepiness detection methods can be invasive or costly.
• Speech analysis offers a non-invasive and cost-effective alternative for monitoring sleepiness.

Purpose of the Study:

• To develop and evaluate a deep transfer learning model for detecting sleepiness using speech data.
• To identify which speech tasks are most effective for sleepiness detection.

Main Methods:

• Utilized the Voiceome dataset with speech from 1,828 participants.
• Developed a deep transfer learning model employing Hidden-Unit BERT (HuBERT) for speech representations.
• Employed masking and separate training techniques to assess the importance of individual speech tasks.

Main Results:

• The best-performing model achieved 80.07% accuracy (0.85 F1-score) using the memory recall task.
• The categorical naming task achieved 81.13% accuracy (0.89 F1-score).
• These tasks from the Boston Naming Test proved crucial for accurate sleepiness detection.

Conclusions:

• Speech analysis, particularly using HuBERT models, is a viable and effective method for detecting sleepiness.
• Specific speech tasks like memory recall and categorical naming are highly indicative of an individual's sleepiness level.
• This research highlights speech as a promising, under-utilized data source for sleepiness monitoring and prevention of adverse events.