Analysis of Interrater Reliability and Interpretive Discrepancies in Polysomnography Scoring Across Clinical...

Area of Science:

• Sleep Medicine
• Diagnostic Accuracy
• Clinical Neurophysiology

Background:

• Polysomnography (PSG) is the gold standard for sleep disorder diagnosis.
• Manual scoring subjectivity leads to inter-scorer variability, impacting accuracy.
• This study quantifies scoring concordance to identify factors causing discrepancies.

Purpose of the Study:

• To quantitatively assess scoring concordance among multiple expert scorers.
• To identify factors contributing to interpretive discrepancies in polysomnography.
• To evaluate scoring reliability across different sleep disorder subgroups.

Main Methods:

• Retrospective analysis of 28,291 epochs from 30 adult PSG studies.
• Three independent scorers evaluated sleep stages, arousals, respiratory events, and leg movements.
• Interrater agreement measured using Fleiss' κ, with subgroup analyses by diagnostic category.

Main Results:

• Overall interrater agreement for sleep staging was nearly perfect (κ = 0.932).
• High reliability observed for respiratory events (apnea κ = 0.955), hypopnea, arousals, and PLMs.
• Severe OSA showed reduced reliability, indicating increased scoring complexity with sleep fragmentation.

Conclusions:

• Expert PSG scoring demonstrates high overall reliability.
• Significant variability persists in complex cases, such as severe OSA.
• Structured quality assurance and automated tools are necessary for diagnostic consistency.