Reliability of a staging assessment system for recurrent respiratory papillomatosis

A staging system for the assessment of severity of disease and response to the therapy in recurrent respiratory papillomatosis (RRP) was proposed several years ago. It includes both a subjective functional assessment of clinical parameters and an anatomic assessment of disease distribution. The anatomic score can then be used in combination with the functional score to measure an individual patient's clinical course and response to the therapy over time. In using this system, it would be of benefit to know what level of variability can be expected from one surgeon to another in the assessment of an RRP patient and the assignment of a score.

Ten videotaped recordings of endoscopic assessments of patients with RRP were reviewed by 15 pediatric otolaryngologists and scored based on the criteria of the staging assessment system.

Analysis was conducted for 15 raters of scoring severity over 25 sites of 10 patients. The total score is the addition of scores over the 25 sites with a score equal to or greater than 20 representing high risk. For 8/10 (80%) of the subjects, there was a complete agreement about risk categorization (low risk) and agreement by 14/15 (93%) raters for categorization of one other patient. For 9/10 (90%) of subjects, the standard errors of the mean total scores were less than 1, meaning a low variance and subsequent high reliability of the total score.

This staging system was able to achieve agreement by 15 pediatric otolaryngologists on 9 of 10 subjects in terms of degree of severity of RRP.