Interobserver agreement and consensus over the esthetic evaluation of conservative treatment for breast cancer

Twenty-four experts from 13 different countries were asked to evaluate photographs taken of 60 women following conservative breast cancer treatment. The esthetic result of each case was classified as poor, fair, good or excellent. Agreement was evaluated using the kappa (k) and weighted kappa (wk) statistics, for all observers, male and female participants, those younger and older than 50 years, those seeing more than 250 cases a year, and those with previous publications in this area. Consensus was obtained by way of a modified Delphi approach, when more than 50% of participants provided the same classification. In a second round, consensual cases were disclosed and a revised opinion was asked in non-consensual ones. Agreement between all participants was fair (k=0.24, wk=0.37) and remained within the same range (k=0.20-0.31, wk=0.31-0.45) in the subgroups analyzed. First round consensus was obtained in 46 out of 60 cases (77%) and in the second round in 59 out of 60 cases (98%). Evaluation of the esthetic results of conservative treatment for breast cancer is only fairly reproducible when performed by experts working in different geographical areas. Consensus is obtainable if a relatively low threshold of agreement is considered acceptable.