Reliability of a Crohn's disease clinical classification scheme based on disease behavior

Classification of Crohn's disease (CD) by disease behavior--either inflammatory (INF), fibrostenotic (FS), or fistulizing/perforating (FP)--has been proposed as a means of assisting management decisions and predicting outcomes for subgroup analysis in clinical trials and for making phenotype/genotype associations in molecular genetic studies. Accurate and reproducible classification of CD patient subgroups is of paramount importance in such studies but to be useful, the classification scheme must have good interrater agreement. We sought to assess the interrater agreement associated with the disease-behavior classification scheme of CD. Twelve patients with CD were randomly selected from a database of 964 patients with CD undergoing medical or surgical treatment or both. Clinical details of the 12 cases, along with their radiographs and surgical and pathological reports, were presented to a panel of 20 experts who were asked to classify each case based on the patient's overall disease course (scenario A) and as if the patient were being entered into a clinical trial on that day (scenario B). Calculations of strength of interrater agreement were made and were expressed as the kappa statistic (kappa), with kappa < 0.2 = poor strength of agreement; kappa 0.21 - 0.4 = fair; kappa 0.41 - 0.6 = moderate; kappa 0.61 - 0.8 = good; and kappa 0.81 - 1.0 = very good. Five panel participants did not complete the study, and three clinical vignettes were excluded because of incomplete scoring, leaving a total of 15 panel experts assessing nine cases. Overall interrater agreement was only fair with kappa = 0.353 for scenario A and kappa = 0.291 for scenario B. Interrater agreement was less when only the most straightforward case in each disease category was evaluated. Classification of CD by pattern of disease behavior yields only fair interrater agreement. This raises concerns regarding its applicability, particularly in ongoing studies of genotype/phenotype associations. Further refinement of disease subtypes and clear operational definitions are required.