Proficiency testing performance: a case study with modeling

Previous literature has approached proficiency testing (PT) performance by defining the minimum levels, and combinations of imprecision and bias, necessary to meet PT requirements. In this case report, current PT performance was assessed and modeling performed to prioritize our quality improvement efforts.

A total of 1,006 chemistry challenge results from Ontario's Laboratory Proficiency Testing Program (LPTP, now QMPLS) performed on 69 tests during 1999 and 2000 were used for this retrospective analysis. Peer group means, all method means and results from reference labs were used for comparison. QMPLS flagging and recommended performance criteria were compiled, and modeling performed to predict different levels of performance.

Our internal imprecision is <5% for 72% of our 69 tests; however, only 20% of our tests had a CV/PT <25%. Of the 1,006 challenges performed, 136 (13.5%) results were outside PT limits, 55 (5.5%) results were flagged, and 12 requests were received from QMPLS seeking clarification on 24 (2.4%) results. Follow-up identified 9 (38%) nonanalytical errors, 8 (33%) method bias errors, 4 (17%) random errors, 2 poor methods, and one with no error identified. Modeling predicted flagging rates of 2.4% using QMPLS recommended precision performance, 1.6% using our current internal imprecision, 2.2% or 7.0% if we included an overall 20% or 50% relative bias rate with our current imprecision levels, or 15.0% when an estimate of our actual bias for each analyte was considered along with our current imprecision levels.

If imprecision were the only cause of PT errors, our flagging rate for this study period would be 1.6%, and we would need to formally investigate 8 results a year. In practice, strict application of the QMPLS PT criteria would result in 68 investigations annually; however, judicial review of the results before request for clarification significantly reduced this number to 12 investigations (of which 38% were nonanalytical errors). At the present time bias is a significant cause of poor PT performance in a variety of assays. Individual laboratories need to address the problem of bias, and ultimately so do manufacturers. It would be helpful if PT programs also acknowledged this necessary evolution in both their criteria and processes.