A quantitative model using mean and standard deviation for evaluation of interobserver agreement in nuclear atypia...

A quantitative model was devised for the purpose of evaluating interobserver agreement in the judgment of the nuclear atypia score of cancer cells in protocol studies. In this model, the mean nuclear atypia score (mu) among observers was considered to approximate the 'true' nuclear atypia score of the tumor, and with an identical mu the strength of interobserver agreement was evaluated by the difference between the ideal standard deviation (sigma) and the observed sigma in the slide conference session. This model was applied to the data of a total of 216 invasive ductal carcinomas acquired at eight slide conference sessions held for the standardization of nuclear atypia scoring and monitoring of interobserver agreement level for enrolled cases in a surgical adjuvant protocol on breast cancer. Compared with conventional percentage interobserver agreement and kappa statistics, the present model, named the lambda model, was more sensitive in detecting the improvement of interobserver agreement in nuclear atypia scoring with repetition of slide conference sessions. The lambda model was particularly effective for evaluating the session which included many intermediate cases with a score between 1 and 2 or between 2 and 3. Percentage interobserver agreement and kappa statistics could estimate the agreement irrespective of the number of observers, but the agreement tended to be underestimated when these intermediate cases were in the majority. The present method was considered to be complementary to conventional methods for evaluating interobserver agreement of cancer histology in protocol studies.