Medical observations and bias

Many clinical and epidemiologic studies are based on different physicians' readings. A method to calibrate the various examinations is described. The standard reader's and others' results are plotted in a correlation diagram. Their readings are characterized by three parameters: frequency of agreement, average difference in size, and variance. In each study, the acceptable range of deviation from the standard reader should be decided. The standard reader is characterized by two requirements: a) that he is consistent, i.e., that a second reading shows a high rate of agreement, a small difference and a small variance between the two readings, and b) that he is reproducible, i.e., that other experienced doctors accept his classification and arrive at the same result. When statistical tests are used, it is understood that systematic errors are not present in the basic data. If this is not the case, the validity of the statistical tests is lost and results distorted or even meaningless. The second part of the study discusses types of bias encountered in epidemiologic studies and gives examples showing that the quantitative significance of bias can be considerable.