Evaluation of Scoring Reliability in Polysomnography at a Single Sleep Center in Thailand

The reliability of Thai certificate qualifications for Advanced Sleep Technicians (ASTs) and Sleep Disorders Specialists (SDSs) in manual polysomnography (PSG) scoring has not been previously evaluated. This study assessed the reliability of PSG scoring performed by ASTs, an SDS, and an automated scoring system (AUTO) at Thammasat University Hospital, Thailand. A retrospective analysis of 250 PSG recordings conducted between September 2022 and February 2023 classified patients into four groups based on the apnea-hypopnea index (AHI): No OSA (AHI <5), mild OSA (AHI 5-15), moderate OSA (AHI 15-30), and severe OSA (AHI >30), comprising 11, 77, 105, and 57 cases, respectively. Scoring reliability was compared among ASTs, SDSs, and AUTO. A single-blinded SDS independently scored the PSG data without knowing the AST's scoring to ensure an unbiased assessment. Across more than 630,000 epochs, the Kappa (κ) statistic demonstrated stronger agreement between AST and SDS (κ = 0.980, 95% CI 0.976-0.984) than between AST and AUTO (κ = 0.599, 95% CI 0.543-0.655), indicating significant differences (p < .0001). For mixed apneas (MAs), intraclass correlation coefficients (ICCs) showed the highest consistency between AST and SDS (ICC = 0.998, 95% CI 0.997-0.998) compared to AST and AUTO (ICC = 0.869, 95% CI 0.836-0.897). Significant differences were observed between AST and SDS compared to AST and AUTO across most metrics (P < .0487). While ASTs and SDSs demonstrated excellent scoring consistency, AUTO scoring was notably less accurate, suggesting that the AUTO system requires further refinement to ensure reliable clinical use.