Related Experiment Video
Updated: Aug 5, 2026
07:40
Multi-Modal Home Sleep Monitoring in Older Adults
Published on: January 26, 2019
Interobserver agreement among sleep scorers from different centers in a large dataset
1Department of Pulmonary and Critical Care Medicine, NYU School of Medicine, New York, NY 10016, USA.
Sleep
|November 18, 2000
Summary
Sleep stage scoring agreement between scorers from different labs averaged 73%, with lower reliability in sleep disordered breathing (SDB) patients. This highlights variability and the need for caution when comparing scored sleep data across institutions.
Area of Science:
- Sleep Medicine
- Neurophysiology
- Biomedical Data Analysis
Background:
- Accurate sleep stage assignment is crucial for diagnosing sleep disorders.
- Inter-scorer reliability can vary, impacting the consistency of sleep study interpretations.
- Previous studies suggest potential differences in scoring based on laboratory protocols.
Purpose of the Study:
- To quantify epoch-by-epoch agreement in sleep stage scoring between scorers from different laboratories.
- To assess the impact of sleep disordered breathing (SDB) on scoring reliability.
- To identify factors contributing to variability in sleep stage assignment.
Main Methods:
- Analysis of 62 nocturnal polysomnograms (NPSGs) from three sleep centers.
- Manual sleep stage scoring by five experienced technologists using standard montage (EEG, EOG, EMG).
- Epoch-by-epoch agreement calculation for overall data, normal sleep, and SDB subsets.
Main Results:
- Mean epoch-by-epoch agreement across all scorers and records was 73%.
- Agreement was higher in normal sleep subsets (mean 76%) compared to SDB subsets (mean 71%).
- 75% of epochs had agreement among at least 4 of 5 scorers; 96% had agreement among at least 3.
Conclusions:
- Sleep stage assignment agreement varies significantly between scorers, diagnoses, and individual records.
- Inter-laboratory agreement is lower than intra-laboratory agreement, necessitating caution in data comparison.
- Reduced agreement in SDB patients supports the notion that sleep fragmentation complicates reliable scoring.
