Meta-audit of laboratory ISO accreditation inspections: measuring the old emperor's clothes

Accreditation to ISO/IEC 17025 is required for EC official food control and veterinary laboratories by Regulation (EC) No. 882/2004. Measurements in hospital laboratories and clinics are increasingly accredited to ISO/IEC 15189. Both of these management standards arose from command and control military standards for factory inspection during World War II. They rely on auditing of compliance and have not been validated internally as assessment bodies require of those they accredit. Neither have they been validated to criteria outside their own ideology such as the Cochrane principles of evidence-based medicine which might establish whether any benefit exceeds their cost. We undertook a retrospective meta-audit over 14 years of internal and external laboratory audits that checked compliance with ISO 17025 in a public health laboratory. Most noncompliances arose solely from clauses in the standard and would not affect users. No effect was likely from 91% of these. Fewer than 1% of noncompliances were likely to have consequences for the validity of results or quality of service. The ISO system of compliance auditing has the performance characteristics of a poor screening test. It adds substantially to costs and generates more noise (false positives) than informative signal. Ethical use of resources indicates that management standards should not be used unless proven to deliver the efficacy, effectiveness, and value required of modern healthcare interventions.