Improving inter-observer agreement and certainty level in diagnosing and grading papillary urothelial neoplasms:...

A Bayesian belief network (BBN), as diagnostic decision support system, enables the processing of our knowledge of histopathology expressed in descriptive terms, words and concepts. The aim of this study was to evaluate the contribution of a BBN in the improvement of inter-observer agreement and certainty level in the diagnosis and grading of papillary urothelial neoplasms.

Inter-observer agreement and certainty level were investigated on 40 cases of non-invasive papillary urothelial neoplasms subdivided according to the WHO 1973 classification. There were 10 urothelial papillomas (UPs), 10 grade 1 papillary carcinomas (G1), 10 grade 2 papillary carcinomas (G2) and 10 grade 3 papillary carcinomas (G3). Five consecutive sessions were held with three observers (RMa, PC and MSt). Sessions A, B and D were based on the morphological evaluation of the specimens with a conventional light microscope only. In sessions C and E, a BBN was used in addition to the microscope. The BBN output was represented by four belief values for four possible diagnostic outcomes. These values ranged from 0.0 to 1.0, with the sum of the belief values being 1.0. Concerning the certainty level, a two-tier system of assessment was adopted in sessions A, B and D: certain versus less certain. In sessions C and E, a belief value equal to or greater than 0.65 was considered as equivalent to "certain".

In session A, an all-encompassing or synthetic approach to decision-making was adopted. Agreement with the gold standard was seen in 60% (RMa), 55% (PC) and 65% (MSt) of cases, respectively. The level of subjective confidence was "certain" in 35%, 40% and 35% of cases, respectively. Better agreement-70% (RMa), 68% (PC) and 72% (MSt) of cases-was present in session B where an analytical approach based on the evaluation of a series of morphological features was used. The level of subjective confidence was "certain" in 45%, 50% and 55% of cases, respectively. In session C, where a BBN was utilised, a further increase in degree of agreement with the gold standard was observed, e.g. 85% (RMa), 80% (PC) and 86% (MSt) of cases, respectively. Levels of certainty or belief values were high. Decrease in both the level of agreement-60% (RMa), 62% (PC) and 65% (MSt) of cases-and certainty was seen in session D where the observers were left free to evaluate the cases morphologically without the constrain of either a synthetic or analytical approach. In session E, where the BBN was used again, the percentage of cases in agreement with the gold standard increased to 83% (RMa), 81% (PC) and 84% (MSt), respectively. Increase in certainty or belief was also seen. The difference of the results obtained in the sessions A, B and D with those seen in the BBN-based sessions (C and E) is statistically significant.

Conventional morphological evaluation of papillary urothelial neoplasms is affected by inter-observer variability and, in many instances, by diagnostic uncertainty. The greatest difficulties are found with G1 and G2 cases. Improvement in inter-observer agreement and certainty level can be achieved with a BBN.