Intra- and inter-rater reliability of endonasal activity estimation in granulomatosis with polyangiitis (Wegener´s)

Granulomatosis with polyangiitis (GPA) frequently starts with an affection of the nasal and paranasal mucosa. Localised GPA of the nasal mucosa or persistent disease activity ('grumbling disease') is often encountered even under immunosuppressive therapy. Necessity for reconstructive surgery is common and careful scheduling to prevent failure and minimise revision rates is crucial. Therefore, reliable estimation of GPA activity in the upper airways using a score is mandatory for diagnosis, follow-up and scheduling reconstructive surgery.

Fifty endoscopic, endonasal images of 45 patients with GPA were used. Twelve (4 German, 8 Mexican) experienced (n=7) and inexperienced (n=5) physicians assessed GPA-activity at two times (T1/T2) in dichotomy and in a grading approach (none, mild, moderate and high activity) using the novel ENT Activity Score (ENTAS). All documents were written in English.

Estimation of activity in dichotomy (none vs. mild/moderate/high): Cohen's Kappa (κ) for intra-rater reliability T1/T2 in inexperienced and experienced physicians was κ=0.58 (agreement 85%) and κ=0.72 (agreement 91%). The inter-rater reliability (Fleiss's κ) T1/T2 for inexperienced and experienced physicians was κ=0.62/κ=0.59 and κ=0.50/κ=0.58 respectively. Estimation of activity in grading approach (none, mild, moderate, high): for inexperienced physicians the intra-rater reliability T1/T2 was κ=0.67 (agreement 56%) and the inter-rater reliability at T1/T2 was κ=0.29 (intraclass correlation coefficient, ICC=0.69) and κ=0.27 (ICC=0.59). For experienced physicians the intra-rater reliability T1/T2 was κ=0.80 (agreement 67%) and the inter-rater reliability at T1 and T2 was κ=0.41 (ICC=0.77) and κ=0.39 (ICC=0.75) respectively.

Intra-rater reliability is high in decision in dichotomy and even in grading activity. There is no difference for experienced or inexperienced physicians. Inter-rater reliability is high in dichotomy, but low for activity grading. Thus, the ENTAS provides a reliable instrument for assessing, documenting and following GPA-related disease activity in the upper respiratory tract. The relationship of activity and following damage needs to be investigated in further studies.