Staging Sleep in Polysomnograms: Analysis of Inter-Scorer Variability

Area of Science:

• Sleep Medicine
• Neuroscience
• Biomedical Engineering

Background:

• Accurate sleep staging during polysomnography (PSG) is crucial for diagnosing sleep disorders.
• Inter-scorer variability in manual sleep scoring is a known challenge, potentially impacting diagnostic reliability.
• Understanding the sources of this variability is essential for developing more consistent scoring methods.

Purpose of the Study:

• To identify the primary reasons for discrepancies among expert scorers in classifying sleep stages from PSG data.
• To quantify the contribution of scorer errors, bias, and difficult-to-classify epochs to overall variability.

Main Methods:

• Fifty-six polysomnograms (PSGs) were manually scored by two experienced technologists, initially and after feedback (M1 and M2).
• An automatic scoring system was used, followed by technologist editing (Edited-Auto), generating six scoring sets per PSG.
• Epochs were categorized as scorer errors, scorer bias, or equivocal (inconsistent scoring) to analyze variability.

Main Results:

• Initial manual scoring agreement was 78.9%, with no significant improvement after scorer feedback and edits.
• Automated scoring with technologist edits showed higher agreement (86.5%).
• Equivocal epochs, characterized by scoring inconsistency, constituted a significant portion (28% ± 12%) of all epochs, driving variability, while scorer errors and bias were minimal (<20% of disagreements).

Conclusions:

• Inter-scorer variability in sleep staging is predominantly driven by the inherent difficulty in classifying certain epochs.
• Scorer errors and bias contribute minimally to disagreements.
• Integrating digitally identified events or calculated variables into the scoring process may substantially reduce variability in polysomnography interpretation.