Do radiologists agree on the quality of computed tomography enterography?

This study aimed to assess variation between radiologists evaluating the quality of multi-detector computed tomography enterography. For 40 consecutive examinations, three experienced radiologists independently rated the following quality variables: % length of adequately filled bowel, bowel lumen diameters, bowel wall delineation, superior mesenteric vein, and bowel wall enhancement, artefacts, and total quality. We calculated the mean difference between observers with standard deviation (SD) for continuous variables and % total agreement, exact Fleiss kappa, and P-values (McNemar's test) for categorical variables. Depending on bowel segment (duodenum distal to bulb, jejunum, ileum, terminal ileum), mean difference between observers ranged from two to 33 (SD from 11 to 32) for % length of adequately filled bowel judged subjectively, 0-2 (SD 0-3) mm for smallest bowel lumen diameter and 0-4 (SD 3-7) mm for largest bowel lumen diameter. Agreement on bowel wall delineation was 80%/kappa 0.50 in duodenum, 90%/kappa 0.57 in jejunum, 75%/kappa 0.14 in ileum and 88%/kappa 0.17 in terminal ileum, where ratings differed between observers (P < 0.04). Agreement was 65%/kappa 0.18 for bowel wall enhancement judged subjectively. For contrast enhancement measured in Hounsfield Units, mean difference between observers ranged from two to 11 (SD 12-15) in normal jejunum wall and zero to one (SD 4-5) in the superior mesenteric vein depending on observer pair. Agreement was 78%/kappa 0.12 for image artefacts. Rating of total examination quality (good/optimal versus poor/very poor) differed between observers (P < 0.01); agreement was 60%/kappa 0.41. Many subjective evaluations varied between observers. We believe that measurements of bowel lumen diameters and contrast enhancement may be preferable.