Automated deep learning-based arousal detection complies with ten expert scorers in unseen data

Area of Science:

• Sleep Medicine
• Artificial Intelligence
• Computational Neuroscience

Background:

• Automatic arousal scoring is crucial for efficient polysomnogram analysis.
• Previous methods faced limitations in dataset homogeneity, scorer agreement, and analysis resolution.
• A robust, generalizable automatic scoring model is needed.

Purpose of the Study:

• To develop and validate a deep learning model for automatic arousal scoring at 1-second resolution.
• To evaluate the model's performance against multiple expert scorers in an independent dataset.
• To assess the model's generalizability across different sleep recordings.

Main Methods:

• A fully convolutional neural network was trained on 1,847 polysomnograms from the Multi-Ethnic Study of Atherosclerosis (MESA) cohort.
• The model was tested on MESA hold-out data and an independent Sleep Revolution (SR) cohort (50 polysomnograms).
• Performance was assessed using event-by-event and 1-second segment analyses against individual scorers and majority agreement.

Main Results:

• In the MESA test set, the model achieved an F1-score of 0.77 (event-by-event) and a κ-value of 0.67 (1-second).
• In the SR dataset, the model's median performance (F1: 0.61, κ: 0.50, ArI ICC: 0.66) surpassed human scorers' median agreement (F1: 0.57, κ: 0.46, ArI ICC: 0.59).
• The model demonstrated superior agreement compared to inter-scorer variability.

Conclusions:

• The deep learning model achieved superior arousal scoring agreement compared to human scorers in an independent dataset.
• The model exhibits strong generalization capabilities for diverse sleep recordings.
• This automated approach offers a reliable tool for objective arousal scoring in clinical and research settings.