Self-Supervised Transformer Model Training for a Sleep-EEG Foundation Model.

Area of Science:

• Neuroscience
• Artificial Intelligence
• Sleep Medicine

Background:

• Current sleep staging (e.g., American Academy of Sleep Medicine's 5 states) offers a limited view of complex sleep neurophysiology.
• There is a need for advanced, data-driven methods to extract deeper insights from neurophysiological sleep signals.

Approach:

• Developed a self-supervised learning model, inspired by masked transformer techniques used in speech processing.
• Trained the model on large-scale neurophysiological sleep data to autonomously learn underlying data structures.

Key Points:

• Self-supervised pre-training demonstrates performance comparable to or exceeding supervised sleep stage classification, particularly with limited labeled data or computational resources.
• The pre-trained model exhibits flexibility, enabling fine-tuning for diverse tasks such as individual identification and "brain age" quantification.

Conclusions:

• Self-supervised learning can automatically extract valuable information potentially missed by conventional sleep staging schemas.
• This approach paves the way for novel sleep classification systems and enhanced data-driven investigations into sleep and brain health.