Inter-test agreement and quantitative cross-validation of immunochromatographical fecal occult blood tests

Immunochromatographical fecal occult blood tests were shown to have higher sensitivity for detecting colorectal neoplasms than the commonly used guaiac-based test. However, positivity rates, sensitivity and specificity vary widely. We aimed to assess the reasons for this heterogeneity. Six dichotomous (qualitative) immunochromatographical tests were used in the same stool samples, taken before cathartic bowel preparation, from 1,330 participants of the German colonoscopy screening program. Positivity rates were determined, and inter-test agreement beyond chance was quantified by kappa coefficients (kappa). In addition, kappa coefficients were expressed in relation to their maximum possible values given differences in test positivity rates (kappa/kappa(max)). Furthermore, the distribution of fecal hemoglobin concentration was assessed by an additional quantitative test in participants classified as clearly positive, borderline positive or clearly negative according to the qualitative tests. Positivity rates strongly varied from 6.4 to 46.8%. As a result, overall agreement between tests was only poor to moderate, with kappa ranging from 0.14 to 0.61. However, apart from the different positivity rates, agreement was mostly very high, with kappa/kappa(max) ranging from 0.53 to 1.00, and exceeding 0.70 in 12 of 15 cases. Distribution of fecal hemoglobin concentrations in the various categories strongly varied across tests. The observed patterns suggest that the strongly different positivity rates essentially reflect different cutoff levels of tests with otherwise very high inter-test agreement. Definition of cutoffs is a critical issue in the application of immunochromatographical tests and should be redefined for several of these tests.