Cohen's kappa for capturing discrimination

Identification of cut-off values for key biomarkers of clinical risk is a useful clinical tool. The kappa coefficient is a popular descriptive statistical measure for summarising the cross classification of two nominal variables with identical classes. On the basis of this definition, I propose that the kappa coefficient can also be used to capture discrimination, in the same way that the receiver operating characteristic (ROC) curve is used in preventive epidemiology studies.

The statistics were determined using Cohen's kappa statistics for a gold standard and a continuous biomarker. The proposed design is compared with the ROC curve by applying it to articles on the metabolic syndrome and a colon cancer clinical trial.

The two methods gave similar results. Moreover, Monte Carlo simulation results confirm that, from a power perspective, the proposed method is to be preferred. In general, the proposed method has higher power than the area under the ROC curve (AUC) for a study of positively correlative design.

Overall, the power performance of the proposed method is better that that of the AUC.