Inter- and intra-rater reliability of two grading systems for equine gastric glandular disease
Related Experiment Video
Updated: Oct 31, 2026
Flexible Colonoscopy in Mice to Evaluate the Severity of Colitis and Colorectal Tumors Using a Validated Endoscopic Scoring System
Published on: October 16, 2013
Claudia Gilders1, Lisa M Katz1, Michael Hewetson2
1UCD School of Veterinary Medicine, University College Dublin, Dublin, Ireland.
Background:
The recommended combined descriptive terminology scoring system for equine gastric glandular disease has poor reliability whereas a published numerical scoring system has substantial reliability. Descriptive terminology appears more affected by clinical experience than numerical scoring. Individual descriptive terminologies have previously correlated with descriptions of severity.
Objectives:
To compare the reliability of two scoring systems across varied clinical experience and between numerical scores and descriptive terms. The hypothesis was that the numerical scoring system would have better reliability regardless of clinical experience and numerical scores would correlate with individual descriptive terms.
Study Design:
Prospective, blinded observer study.
Methods:
Duplicate gastroscopy images of 30 glandular lesions were assessed by 12 raters using both scoring systems; raters were split into four groups: internal medicine specialists, residents, general practitioners and new graduates. Intra- and inter-rater reliability of both systems and individual criteria therein were determined using Krippendorff's alpha.
Results:
Intra and inter-rater reliability were higher for the numerical scoring system when assessing all raters combined (α = 0.89, 95% CI 0.86-0.90 and α = 0.67, 95% CI 0.65-0.69 respectively) compared to the combined descriptive terminology system (α = 0.61, 95% CI 0.55-0.66 and α = 0.09, 95% CI 0.08-0.10 respectively). Intra-rater reliability was high for all groups with the numerical system, but only high for specialists with the descriptive system. Intra-rater reliability was more affected in the descriptive scoring system by clinical experience than the numerical scoring system. Inter-rater reliability was moderate for specialists, residents and recent graduates with the numerical system, but poor for all groups with the descriptive system. There was a large and statistically significant association between numerical score and the score for 'severity' (η2_H = 0.66, 95% CI 0.52-0.75; p < 0.001).
Main Limitations:
Use of still images visualised on individual screens.
Conclusions:
The numerical scoring system was more reliable than the descriptive system regardless of clinical experience. Numerical score and 'severity' are strongly associated.
