Reliability and initial validation of the ulcerative colitis endoscopic index of severity

We studied the reliability of the previously described Ulcerative Colitis Endoscopic Index of Severity (UCEIS) and validated it with an independent cohort of investigators.

We created a new library of 57 videos of flexible sigmoidoscopy and stratified them based on disease severity. Twenty-five investigators were each randomly assigned to assess 28 videos (which included 4 duplicates to assess intraobserver reliability). Investigators were blinded to clinical details except for 2 of 4 duplicated videos (to assess the impact of knowledge of symptoms on assessment). Three descriptors ("vascular pattern", "bleeding", and "erosions and ulcers") comprising the UCEIS were scored with a visual analogue scale (VAS) to assess overall severity. Intrainvestigator and interinvestigator agreement was characterized by κ statistical analysis; reliability ratios were used to compare VAS and UCEIS scores.

There was a high level of correlation between UCEIS scores and overall assessment of severity (correlation coefficient, 0.93). Internal consistency (Cronbach α analysis) was 0.86. Intrainvestigator and interinvestigator reliability ratios for UCEIS scores were 0.96 and 0.88, respectively. Intrainvestigator agreement in determination of the UCEIS score was good (κ = 0.72), with individual descriptors ranging from a κ of 0.47 (for bleeding) to 0.87 (for vascular pattern). Interinvestigator agreement in determination of UCEIS scores was moderate (κ = 0.50), with descriptors ranging from a κ of 0.48 (for bleeding) to 0.54 (for vascular pattern). Intrainvestigator variability in determining UCEIS scores did not change appreciably when a video was presented with clinical details.

The UCEIS and its components show satisfactory intrainvestigator and interinvestigator reliability. Among investigators, the UCEIS accounted for a median of 86% of the variability in evaluation of overall severity on the VAS when assessing the endoscopic severity of UC and was unaffected by knowledge of clinical details.