A statistical procedure for measuring and evaluating performance in interlaboratory comparison programs

There are scientific and regulatory needs to measure individual laboratory performance on a series of challenges, for single analytes and for all analytes in a particular discipline. Because these needs must be met with a very limited amount of information, optimal statistics should be used to measure performance. Since punitive action could result from poor performance, there should be precise quantitative goals that can be measured directly with the performance statistic. Finally, it is important to limit the likelihood of falsely penalizing a laboratory, since the large majority of laboratories are not in need of regulatory action. A statistic is described that measures individual performance on quantitative interlaboratory proficiency tests. This statistic is based on actual error relative to the amount of error that is tolerable. It can be accumulated over several challenges (specimens tested as unknowns) to give an estimate of a participant's performance level for that analyte. It can also be accumulated across analytes to give scores for Survey mailings or for accumulated performance. Because the statistic measures error, it contains more information than does the percentage of acceptable results, and therefore has greater power to detect poor performance. The distribution of the statistic is described and tested for validity. Then, a procedure is presented to evaluate laboratories relative to a performance goal. The entire procedure is then tested with recent College of American Pathologists Chemistry Survey data.