Inter-rater reliability of published flow diversion occlusion scales

With increasing use of flow-diverting stents for the treatment of intracranial aneurysms, standardized methods and a common language to evaluate angiographic outcomes are needed. Multiple grading scales have been developed for this purpose but none has been widely adopted.

To analyze these scales to determine interobserver reliability.

Four independent assessors scored the intraprocedural angiograms of patients who underwent flow-diverting stent deployment for an intracranial saccular or fusiform aneurysm at our institution between October 2012 and June 2015. Angiographic outcome immediately after flow-diverting stent deployment was scored using three grading scales (Kamran-Byrne (KB), Simple Measurement of Aneurysm Residual after Treatment (SMART), and O'Kelley, Krings, Marotta (OKM)). Statistical analysis was performed using Light's κ for multiple raters (κ), Kendall's coefficient of concordance (W), and intraclass correlation (ICC).

We included the angiograms of 50 consecutive patients (mean age 58 years, range 30-79) who underwent flow-diverting stent deployment for an intracranial aneurysm (40 saccular, 10 fusiform). Six aneurysms were located in the posterior circulation. The inter-rater reliability was typically poor or fair: SMART aneurysm filling (κ=0.30, W=0.36, ICC=0.12), SMART parent vessel stenosis (κ=0.07, W=0.33, ICC=0.12), KB axis I (κ=0.24, W=0.50, ICC=0.25), KB axis II (κ=0.07, W=0.30, ICC=0.06), OKM aneurysm filling (κ=0.23, W=0.45, ICC=0.13), OKM contrast stasis (κ=0.36,W=0.71, ICC=0.54).

Existing flow-diverting stent grading scales have low inter-rater reliability for most categories.