Inter- and Intra-Rater Reliability of 4D-CT Versus Bronchoscopy in Diagnosing and Grading Tracheomalacia

Bronchoscopy and 4D-CT can be used to diagnose and characterize tracheomalacia. This study aims to assess inter- and intra-rater reliability of rigid bronchoscopy and 4D-CT imaging in determining tracheomalacia severity.

De-identified rigid bronchoscopy videos and 4D-CT derived airway segmentations were collected for 10 patients with varying degrees of dynamic tracheal collapse, for a total of 20 videos for review. A randomized, multidisciplinary survey was administered to physicians from otolaryngology, pulmonology, general surgery, and radiology at two institutions. Participants rated the severity of collapse for all cases at two time points, 1 month apart. Gwet's AC2 was used to assess inter- and intra-rater reliability, with values ranging from 0 (slight agreement) to 1 (almost perfect agreement).

Nineteen physicians completed the initial ratings (T₁), and 12 completed the repeat ratings 1 month later (T₂). Inter-rater agreement for bronchoscopy was substantial at T₁ (AC2 = 0.79, 95% CI: [0.57, 1]) and at T₂ (AC2 = 0.76, 95% CI [0.52, 1]). Similarly, ratings of 4D-CT segmentations demonstrated substantial agreement initially (T₁: AC2 = 0.66, 95% CI [0.44, 0.89]) and at T₂ (AC2 = 0.68, 95% CI [0.39, 0.97]). Bronchoscopy (AC2 = 0.91, 95% CI: [0.87, 0.96]) and 4D-CT (AC2 = 0.84, 95% CI: [0.76, 0.91]) had substantial agreement for intra-rater reliability.

Endoscopic assessment showed higher inter- and intra-rater reliability than 4D-CT. Future studies should focus on defining the optimal role of these modalities to establish a standardized severity rating, thereby improving identification of surgical candidates.