Inter-rater reliability in diagnosis and treatment of type one laryngeal cleft: A blinded observational study

Type one laryngeal cleft (T1LC) has been implicated as a major contributor to aspiration in non-syndromic pediatric dysphagia. Despite an increasing incidence, there remains controversy in diagnosis and treatment algorithms.

The primary objective of this study was to evaluate the inter-rater reliability (IRR) for the diagnosis and treatment of T1LC.

A retrospective analysis was conducted to identify children evaluated for a T1LC from 2016 to 2017 at a single tertiary care center. The microlaryngoscopy video recordings depicting palpation of the interarytenoid region with a right-angle probe were reviewed. These recordings were shown to blinded pediatric otolaryngologists and each surgeon's determination of the presence or absence of a T1LC as well as recommended treatment (observation, injection laryngoplasty, or endoscopic cleft repair) was recorded and compared against the other blinded surgeons. Fleiss's kappa was calculated to evaluate IRR in both diagnosis and treatment.

Eight pediatric otolaryngologists were included in the study with a mean post-training experience of 15 years (range 1-35 years). The inter-rater percent agreement in diagnosis of our patient population was 28.6% (range 3.7-71%) with a kappa value of 0.31 (p < 0.0001). In regard to management, the inter-rater percent agreement in treatment was 11.4% (range 0-35%) with a kappa value of 0.14 (p = 0.01).

This study highlights the challenges and variation that exists among surgeons in diagnosing and managing potential T1LC. Further standardizing the endoscopic examination and treatment algorithm may reduce diagnostic and treatment discordance.