Statistical evaluation of method-comparison data

Comparison of two methods for determining a substance is a common procedure in clinical chemistry. Almost without exception, normal theory statistics are used with the tacit assumption that the data under test show gaussian distributions. Experience has led us to question this assumption for populations of differences between values obtained by two methods applied to common samples and for populations of paired values for each method as used in correlation analysis. Data are presented from our laboratory to illustrate these points, as well as an examination of data from relevant papers that appeared in this journal from 1968 through 1973. These examples indicate that incorrect interpretation of results because of insufficient attention to non-gaussian distribution is a real and present danger. We present a systematic approach to the statistical aspect of comparing methods that enables one to avoid this problem.