Self-Supervised Transformer Model Training for a Sleep-EEG Foundation Model.

Area of Science:

• Neuroscience
• Artificial Intelligence
• Sleep Medicine

Background:

• The American Academy of Sleep Medicine (AASM) uses a 5-class sleep/wake state schema (Wake, N1, N2, N3, REM).
• This schema offers a high-level overview but may miss crucial neurological or health information within sleep signals.
• Data-driven methods are needed to analyze complex sleep neurophysiology.

Purpose of the Study:

• To develop a self-supervised learning approach for analyzing neurophysiological sleep data.
• To investigate if this method can uncover information beyond the standard AASM sleep staging.
• To assess the model's flexibility and performance on novel tasks and data.

Main Methods:

• Utilized a masked transformer training procedure, inspired by speech transcription models.
• Applied self-supervised pre-training on large quantities of neurophysiological sleep data.
• Evaluated the model's performance against supervised sleep stage classification and fine-tuned it for new tasks.

Main Results:

• Self-supervised pre-training matched or surpassed supervised sleep stage classification accuracy, particularly with limited data or computational resources.
• The pre-trained model demonstrated flexibility, performing well on unseen EEG recording montages.
• The model successfully adapted to new tasks, including individual identification and brain age quantification.

Conclusions:

• Modern self-supervised methods can automatically learn valuable information from sleep data that may be overlooked by the current 5-class AASM schema.
• This approach lays the foundation for developing novel sleep scoring systems.
• The findings support further data-driven exploration of sleep for uncovering new biomarkers and understanding sleep's complexity.