Bristol Stool Form Scale reliability and agreement decreases when determining Rome III stool form designations

Rater reproducibility of the Bristol Stool Form Scale (BSFS), which categorizes stools into one of seven types, is unknown. We sought to determine reliability and agreement by individual stool type and when responses are categorized by Rome III clinical designation as normal or abnormal (constipation or diarrhea).

Thirty-four gastroenterology providers from three institutions rated 35 stool photographs using the BSFS. Twenty rerated the photographs.

1190 individual stool type ratings were completed. Though only four photographs had absolute agreement (all Type 1 or Type 7), general agreement was high with 1132 (95.1%) of ratings being within one category type of the modal rating. Inter-rater and intra-rater reliability of the BSFS by individual stool type was excellent with intraclass correlations of 0.88 (95% CI: 0.86-0.90, p < 0.001) and 0.89 (95% CI: 0.86-0.91, p < 0.001), respectively. However, agreement decreased when using Rome III designations with 13 (37%) photographs having significantly diverging classifications (semi-interquartile range = 0.5). These 13 photographs were rated by the majority of raters as either type 2 vs type 3 or type 5 vs type 6 stools, representing the boundaries of normal vs abnormal stools. Inter-rater and intra-rater reliability of the BSFS by Rome III clinical categorization decreased with intraclass correlations of 0.75 (95% CI: 0.69-0.81, p < 0.001) and 0.65 (95% CI: 0.49-0.81, p < 0.001), respectively.

The Bristol Stool Form Scale has excellent reliability and agreement when used to rate individual stool type by raters. However, BSFS reliability and agreement decreases when determining Rome III stool form categories.