A comparative analysis of automatic and manual scoring methods in polysomnography

Area of Science:

• Sleep Medicine
• Medical Technology

Background:

• Polysomnography (PSG) is crucial for diagnosing sleep disorders.
• Manual scoring (MS) by experienced technicians is the gold standard, but time-consuming.
• Automatic scoring (AS) software offers potential efficiency gains but requires validation.

Purpose of the Study:

• To compare 26 PSG parameters between automatic scoring (AS) software and manual scoring (MS) techniques.
• To evaluate the reliability of experienced technicians in PSG scoring.
• To assess the concordance between different scoring methods.

Main Methods:

• Three hundred sixty-nine PSG records were independently scored by two experienced manual scoring (MS) groups and one automatic scoring (AS) software.
• Comparison of 26 PSG parameters across AS, MS group 1, and MS group 2.
• Statistical analysis included Kendall's tau-B for agreement and p-values for significance.

Main Results:

• No significant differences were found between the two manual scoring (MS) groups for any PSG parameter.
• Automatic scoring (AS) differed significantly from both MS groups in several parameters, including sleep stages and apnea-hypopnea index (p < 0.05).
• Strong agreement was observed between MS groups (Kendall's tau-B = 0.747-0.922), while AS showed poor correlation with MS for apnea/hypopnea counts.

Conclusions:

• Automatic scoring (AS) software yields imprecise results for numerous PSG parameters compared to experienced manual scoring (MS).
• AS should not be the exclusive scoring method until AI algorithms improve.
• PSG scoring by technicians with at least five years of experience is sufficient and dependable.