Comparison of different approaches to evaluate External Quality Assessment Data

Area of Science:

• Laboratory Science
• Statistical Analysis

Background:

• Z-scores are crucial for evaluating laboratory performance in External Quality Assessment (EQA) programs.
• Identifying poorly performing laboratories, even with outliers, is essential for EQA program integrity.

Purpose of the Study:

• To compare the effectiveness of different Z-score calculation approaches in detecting laboratory outliers.
• To determine which statistical method performs best in identifying true outliers while minimizing false positives.

Main Methods:

• Simulated data from Normal and Student t distributions were used, with some data intentionally contaminated with outliers.
• Outlier-based (e.g., Grubbs, Dixon) and robust (e.g., Tukey, Qn, ISO) methods were assessed.
• Performance was evaluated by recording true and false outlier rates and deriving Positive and Negative Predictive Values.

Main Results:

• All tested methods performed similarly with sample sizes >10 and well-separated outliers.
• For smaller sample sizes and closer outliers, performance varied significantly.
• Tukey's method had high true and false outlier rates; ISO and Qn showed weak performance.
• Grubbs test demonstrated the best overall results in identifying true outliers.

Conclusions:

• Grubbs test is recommended for EQA programs due to its superior performance in outlier detection.
• The choice of Z-score method significantly impacts laboratory performance evaluation, particularly with challenging datasets.