Interobserver agreement among clinicians in the treatment recommendations for patients with unruptured intracranial...

In practice, variability in clinical decisions among clinicians can exist and may be acceptable. However, from the patient's standpoint, particularly for those with potentially life-threatening conditions such as unruptured intracranial aneurysms (UIAs), this variability can be confusing and concerning. In this study, the levels of interobserver agreement in clinical decisions among three endovascular neurosurgeons working at the same hospital were analyzed using a newly developed grade of recommendation (GOR) system.

A total of 161 consecutive patients with 202 UIAs were included in this study. The GOR system consists of six grades, representing various clinicians' recommendations, including treatment or no-treatment. The three observers reviewed the medical records and digital subtraction angiography images, and then assigned the corresponding GORs for each aneurysm. Interobserver agreement was analyzed using Fleiss' kappa values.

The overall Fleiss' kappa among three observers was 0.52, indicating a moderate level of interobserver agreement. The agreement was relatively high for grades 1 and 5. It was the lowest for grade 3. When GORs were classified as treatment, middle and no-treatment groups, the overall kappa value was 0.84, indicating almost perfect.

This study shows that the level of interobserver agreement was very high for treatment versus no-treatment decisions, but moderate regarding the strength of the recommendations. Further studies are needed to clarify the detailed reasons for the similarities and differences in clinicians' recommendations.