External validation of the Chicago Chiari Outcome Scale

Historically, assessment of clinical outcomes following surgical management of Chiari malformation Type I (CM-I) has been challenging due to the lack of a validated instrument for widespread use. The Chicago Chiari Outcome Scale (CCOS) is a novel system intended to provide a less subjective evaluation of outcomes for patients with CM-I. The goal of this study was to externally validate the performance of the CCOS.

Patients undergoing surgery for CM-I between 2001 and 2012 were reviewed (n = 292). Inclusion criteria for this study were as follows: 1) patients receiving primary posterior fossa decompression; 2) at least 5.5 months of postoperative clinical follow-up; and 3) patients ≤ 18 years of age at the time of surgery. Outcomes were evaluated using the CCOS, along with a "gestalt" impression of whether patients experienced significant improvement after surgery. A subgroup of 118 consecutive patients undergoing operations between 2008 and 2010 was selected for analysis of interrater reliability (n = 73 meeting inclusion/exclusion criteria). In this subgroup, gestalt and CCOS scores were independently determined by 2 reviewers, and interrater reliability was assessed using the intraclass correlation coefficient (ICC) and kappa (κ) statistic.

The median CCOS score was 14, and 67% of patients had improved gestalt scores after surgery. Overall, the CCOS was effective at identifying patients with improved outcome after surgery (area under curve = 0.951). The interrater reliability of the CCOS (ICC = 0.71) was high, although the reliability of the component scores ranged from poor to good (ICC 0.23-0.89). The functionality subscore demonstrated a low ICC and did not add to the predictive ability of the logistic regression model (likelihood ratio = 1.8, p = 0.18). When analyzing gestalt outcome, there was moderate agreement between raters (κ = 0.56).

In this external validation study, the CCOS was effective at identifying patients with improved outcomes and proved more reliable than the authors' gestalt impression of outcome. However, certain component subscores (functionality and nonpain symptoms) were found to be less reliable, and may benefit from further definition in score assignment. In particular, the functionality subscore does not add to the predictive ability of the CCOS, and may be unnecessary. Overall, the authors found the CCOS to be an improvement over the previously used assessment of outcome at their institution.