Related Experiment Video
Updated: May 2, 2026
Comprehensive Endovascular and Open Surgical Management of Cerebral Arteriovenous Malformations
Published on: October 20, 2017
Observer reliability of arteriovenous malformations grading scales using current imaging modalities
Christoph J Griessenauer1, Joseph H Miller, Bonita S Agee
1Department of Neurosurgery and.
Object:
The aim of this study was to examine observer reliability of frequently used arteriovenous malformation (AVM) grading scales, including the 5-tier Spetzler-Martin scale, the 3-tier Spetzler-Ponce scale, and the Pollock-Flickinger radiosurgery-based scale, using current imaging modalities in a setting closely resembling routine clinical practice.
Methods:
Five experienced raters, including 1 vascular neurosurgeon, 2 neuroradiologists, and 2 senior neurosurgical residents independently reviewed 15 MRI studies, 15 CT angiograms, and 15 digital subtraction angiograms obtained at the time of initial diagnosis. Assessments of 5 scans of each imaging modality were repeated for measurement of intrarater reliability. Three months after the initial assessment, raters reassessed those scans where there was disagreement. In this second assessment, raters were asked to justify their rating with comments and illustrations. Generalized kappa (κ) analysis for multiple raters, Kendall's coefficient of concordance (W), and interclass correlation coefficient (ICC) were applied to determine interrater reliability. For intrarater reliability analysis, Cohen's kappa (κ), Kendall's correlation coefficient (tau-b), and ICC were used to assess repeat measurement agreement for each rater.
Results:
Interrater reliability for the overall 5-tier Spetzler-Martin scale was fair to good (ICC = 0.69) to extremely strong (Kendall's W = 0.73) on initial assessment and improved on reassessment. Assessment of CT angiograms resulted in the highest agreement, followed by MRI and digital subtraction angiography. Agreement for the overall 3-tier Spetzler-Ponce grade was fair to good (ICC = 0.68) to strong (Kendall's W = 0.70) on initial assessment, improved on reassessment, and was comparable to agreement for the 5-tier Spetzler-Martin scale. Agreement for the overall Pollock-Flickinger radiosurgery-based grade was excellent (ICC = 0.89) to extremely strong (Kendall's W = 0.81). Intrarater reliability for the overall 5-tier Spetzler-Martin grade was excellent (ICC > 0.75) in 3 of the 5 raters and fair to good (ICC > 0.40) in the other 2 raters.
Conclusion:
The 5-tier Spetzler-Martin scale, the 3-tier Spetzler-Ponce scale, and the Pollock-Flickinger radiosurgery-based scale all showed a high level of agreement. The improved reliability on reassessment was explained by a training effect from the initial assessment and the requirement to defend the rating, which outlines a potential downside for grades determined as part of routine clinical practice to be used for scientific purposes.
Insights
Observer reliability for arteriovenous malformation (AVM) grading scales like Spetzler-Martin and Spetzler-Ponce showed good agreement, with the Pollock-Flickinger scale demonstrating excellent reliability. Reassessment improved accuracy, suggesting a training effect.
Area of Science:
- Neurosurgery
- Radiology
- Medical Imaging
Background:
- Arteriovenous malformations (AVMs) require accurate grading for treatment planning.
- Existing grading scales (Spetzler-Martin, Spetzler-Ponce, Pollock-Flickinger) are widely used but their observer reliability needs evaluation.
- Current imaging modalities and clinical practice settings are crucial for assessing scale reliability.
Purpose of the Study:
- To evaluate the observer reliability of three common arteriovenous malformation (AVM) grading scales.
- To compare the reliability of the Spetzler-Martin (5-tier), Spetzler-Ponce (3-tier), and Pollock-Flickinger scales.
- To assess reliability using current imaging techniques (MRI, CT angiography, DSA) in a clinical context.
Main Methods:
- Five experienced raters independently graded 15 MRI, 15 CT angiograms, and 15 DSA studies of AVMs.
- Intrarater reliability was assessed by repeating assessments on 5 scans per modality.
- Interrater reliability was analyzed using generalized kappa, Kendall's W, and ICC; intrarater reliability used Cohen's kappa, Kendall's tau-b, and ICC.
Main Results:
- Good to extremely strong interrater reliability was observed for Spetzler-Martin (ICC=0.69, W=0.73) and Spetzler-Ponce (ICC=0.68, W=0.70) scales, improving upon reassessment.
- Excellent to extremely strong interrater reliability was found for the Pollock-Flickinger scale (ICC=0.89, W=0.81).
- CT angiography showed the highest agreement, followed by MRI and DSA. Intrarater reliability for Spetzler-Martin was excellent in 3/5 raters.
Conclusions:
- All three AVM grading scales demonstrated high levels of agreement among observers.
- Reassessment improved reliability, likely due to a training effect and the requirement for justification.
- The need for justification in routine clinical practice may impact the utility of grades for scientific research.
More Related Videos
13:16Computed Tomography and Optical Imaging of Osteogenesis-angiogenesis Coupling to Assess Integration of Cranial Bone Autografts and Allografts
Published on: December 22, 2015
08:52Software-Assisted Quantitative Measurement of Osteoarthritic Subchondral Bone Thickness
Published on: March 18, 2022
Related Concept Videos
01:19Imaging Studies VII: Vascular Imaging
01:25Imaging Studies for Cardiovascular System VI: Calcium -Scoring CT