Agreement Statistics for Methods Comparison Based on Total Error Per CLSI EP46 and FDA Assay Criteria Using Web Tool

Per CLSI EP46 (2025), the validation of a test device, when compared to a reference device, should focus on evaluating the total analytical error (TAE) and show that it is within an allowable total error (ATE). Specifically, FDA Assay criteria (2013) suggest that 95% of test device data should be within ATE from those of a reference device. This article presents statistical tools for assessing agreement in method comparisons and assay validations that satisfy the above guidance. Here, we emphasize the concept and provide examples using a web tool.

The validated web tool, free for public use, is available to satisfy the above hard-defined criteria for device acceptance and provides information regarding accuracy and precision. We use test and reference measurements of selected clinical chemistry and hematology analytes as examples. We selected 4 analyte cases to evaluate: precise and accurate, precise and inaccurate, imprecise and accurate, and imprecise and inaccurate.

Among the 4 analytes, the precise and accurate case is the only one that passed the FDA Assay criteria. We also present the misleading results indicated by traditional paired t-test and regression analyses, which rejected the case that passed the FDA Assay criteria.

The freely accessible and validated web tool successfully identifies analytes that meet the FDA Assay criteria and provides an indication of when an investigation of a potential precision and/or accuracy problem is required.