Inter-observer variation in decision-making regarding patients on chronic hemodialysis: a study using the kappa index

To examine inter-observer variation in the monitoring of patients with chronic disease, four nephrologists independently assessed 62 patients on maintenance hemodialysis. Deviation from normal was determined for adequacy of dialysis, protein intake, and metabolic state. The kappa-index, which allows to adjust for chance agreement, was used to analyze each monitoring diagnosis. Low agreement was found on decisions concerning adequacy of dialysis (kappa 0.12-0.26), while agreement was higher about protein intake (kappa 0.21-0.46), and metabolic state (kappa 0.24-0.52). Two physicians classified no patient as overdialyzed, while 16-18% were thus categorized by the other two. Routines for review of recent medical history also differed significantly between the physicians. Measures are needed to increase the reliability of decisions regarding the monitoring of chronic hemodialysis. A long lasting physician-patient relationship is not a sufficient prerequisite for diminishing decision variation. Medical audit as part of the clinical routine, and use of additional sources of information, exemplified by urea kinetic modeling, are discussed.