A simple, graphical method to evaluate laboratory assays

Evaluation methods of laboratory assays often fail to predict the large, infrequent errors that are a major source of clinician complaints. We present a simple, graphical method to evaluate laboratory assays, which focuses on detecting large, infrequent errors. Our method, the folder empirical cumulative distribution plot or, more simply, mountain plot, is prepared by computing a percentile for each ranked difference between the new and reference method. To get a folded plot, one performs the following subtraction for all percentiles over 50: percentile = 100 - percentile. Percentiles (y axis) are then plotted against differences or percent differences (x axis). The calculations and plots are simple enough to perform in a spreadsheet. We also offer Windows based software to perform all calculations and plots. The mountain plot compared to the difference plot focuses attention on two features of the data: the center and the tails. We prefer the mountain plot over other graphical techniques because: 1. It is easier to find the central 95% of the data. 2. It is easier to estimate percentile for large differences (e.g., percentiles greater than 95%). 3. Unlike a histogram, the plot shape is not a function of the intervals. 4. Comparing different distributions is easier. 5. The plot is easier to interpret than a standard empirical cumulative distribution plot. Difference and mountain plots each provide complementary perspectives on the data. We recommend both plots. This method can also be used with data from a wide variety of other applications such as clinical trials and quality control.